Improve data quality management and make sure the data integration into Cognite Data Fusion (CDF) data sets is reliable by monitoring extraction pipelines. If any data integration issues occur, you can be quickly notified, start troubleshooting, and contact the pipeline owner and other stakeholders.Documentation Index
Fetch the complete documentation index at: https://docs.cognite.com/llms.txt
Use this file to discover all available pages before exploring further.
View extraction pipeline details
Navigate to extraction pipelines
Navigate to Data management.
- To view all extraction pipelines across data sets in a CDF project and their latest status, navigate to Integrate > Extraction pipelines.
- To view extraction pipelines for the selected data set, navigate to Data catalog. Then select a data set, and open the Lineage tab.
Understanding status and uptime
The Extraction pipeline statuses reflect the latest known state of the pipeline and its runs. Treat the status as a signal to investigate. One failing step, such as authentication, connectivity, scheduling, or a failed upload to CDF, can show Failure or otherwise look unhealthy even when the rest of your integration is still healthy. Choose the tab that matches your extractor.- On-premises extractors
- Hosted extractors
On Overview, Last reported status summarizes successful and failed critical operations. Last connected shows when an extractor last communicated with CDF, and Schedule shows when the extractor is expected to run.On Run history, the Message column shows the errors or success summaries. Compare timestamps and Configuration revision across runs. Treat any failing status or stale timestamp as a signal to investigate.
Troubleshooting failed integrations
If an extraction pipeline fails or a continuous data flow into CDF is interrupted, you’ll get an email from the system (if activated in setup). Select the email link to open the Extraction pipeline overview. Choose the tab that matches your extractor.- On-premises extractors
- Hosted extractors
- Select the pipeline name to open the Extraction pipeline overview tab to find documentation and key properties.
- Use Owner and Contacts to see who to inform about a failure.
- Review and confirm what is happening with Last reported status, Last run time, and Last connected.
- Switch to the Run history tab to review past runs, filter by time and Last run status.
- Review the chart, extractor metrics, and logs to get broader knowledge to see how the extraction process has performed over time.
- When you have fixed the failure, update the pipeline documentation with any lessons learned.