Aller au contenu principal

About data extraction

Extractors connect to source systems and push data in its original format to Cognite Data Fusion (CDF) as part of the data integration workflow. You can extract data with prebuilt Cognite extractors or create a customer extractor using Python and .NET utils packages and SDKs.

The prebuilt extractors can stream or extract data in batches to the CDF staging area or directly to a CDF service with little or no data transformation.

Note

The Cognite extractors only need read access to the source systems and never modify the original data.

You can also ingest data into CDF using the Cognite API or integrate ETL tools with the PostgreSQL gateway if your data is already in the cloud.

Integration architecture

Cognite extractors

ExtractorDescription
Cognite DB extractorConnects to databases using ODBC, executes queries, and batch extracts data into the CDF staging area or directly to the CDF time series service. Make sure you have ODBC drivers for the databases you're connecting to. This extractor is available as both a Windows service and a standalone executable.
Cognite Documentum extractorConnects to OpenText Documentum and Documentum D2 and batch extracts documents via the Documentum Foundation Classes (DFC) library or the D2 REST API (recommended) into the CDF files service. It extracts metadata into the CDF staging area.
Cognite PI extractorConnects to the PI Data archive and streams time series into the CDF time series service. In parallel, the extractor ingests historical data (backfilling).
Cognite OPC UA extractorConnects to the open OPC UA protocol and streams time series into the CDF time series service and events into the CDF events service. It batch extracts the OPC UA node hierarchy into the CDF staging area or as CDF assets and relationships.
Cognite PI AF extractorConnects to the PI Asset Framework and batch extracts Asset Framework elements and all their attributes. The extractor then builds a tree for each element in the CDF staging area.

Download installation files

  1. Navigate to Cognite Data Fusion, and select Extract data.

  2. Select the extractor best suited to your source system.

  3. Download the Installation file and, optionally, the PDF documentation.

    Extractor download page

Custom extractors

If your source system cannot use any of the prebuilt extractors, you can create custom extractors. Select Create a custom extractor to find Python and .NET utils packages and SDKs to get started.

Cognite Python extractor-utils SDK

Use the Python extractor-utils SDK, which is an extension of the Cognite Python SDK, to develop extractors in Python. For more information, see the Python extractor-utils SDK documentation.

Cognite .NET extractor utils code

Use the .NET extractor to develop extractors in C#. For more information, see the .NET utils documentation.