Data sources and integration

To get started, we recommend that you focus on well-defined data sets for the asset/equipment in your scope and on the essential data to solve the business problem. Often, it’s helpful to start with data sets that have an immediate integration option available. Beginning with a relatively straight-forward ingestion pipeline can provide immediate project benefits. For example, you discover how the team best works together, and you can quickly showcase CDF’s visualization capabilities to business and application users. Visualization is a potent tool for data exploration and often generates further excitement and interest in your CDF project. It’s common first to integrate one or more of these types of data:

Asset hierarchy (or equipment hierarchy if applicable). This is often built using a data dump.
Process data (time series).
Work orders or alarms.
3D/photogrammetry models of assets integrated into the CDF data model (at the moment the Asset Hierarchy).
Documents associated with an asset integrated into the CDF data model (at the moment the Asset Hierarchy).

These questions can help you plan your integration:

Key questions	Insights
At what frequency does the data need to be updated (for example, live vs. batch)?	Understand the business requirements for the update frequency and data freshness.
What is the data source, and who is the system administrator?	Figure out where and in what type of system the data is stored.
Is the data available, and how is it being used today?	Contact source system administrators and data consumers to learn more.
Is the data in the source system what you expected?	Get a data dump from the source system. Verify with data consumers and subject matter experts that the data meets the requirements to build your solution.
Do you understand the data model in the source system?	Read the source system data model’s technical documentation, or find a technical expert who can answer data questions.
What are the integration options? Are there any existing connection interfaces available for the source? Are there any existing connectors in place for the system? Can I use existing connectors, or do I need to build a new? What connectivity, access, and infrastructure is required to extract data from the source system? Are there security principles for the data source in scope (for example, types of users, access, etc.)?	Work closely with the enterprise architect and the infrastructure and security engineers to determine your business’s optimal integration options. Get approval for system architecture and data source access. Identify if you can use existing or if you have to build new ones. You can find information about our standard extractors here.
When can the integration be finished?	Align on a team plan and the deliverables necessary for setting up the live integrations.