What extraction pipelines do
An extraction pipeline defines a data flow from a source system to CDF. You create pipelines to track which extractors or custom integrations are running, how often they run, and whether they succeed or fail. Pipelines are the parent objects for runs and configurations.Visibility and governance
You can create, update, and monitor extraction pipelines to track data flow from source systems. Use pipelines alongside data sets and labels to document lineage and governance. Pipeline runs record each execution for audit and troubleshooting.Extraction pipelines are metadata objects. The actual data extraction is performed by extractors or custom code that reports to the pipeline.
Key capabilities
- Create and update extraction pipelines
- Monitor health and status of data ingestion
- Link to runs for execution history
- Govern configuration through the config API