CDF Entity Matching reference

​

Who it’s for

• Data engineers — deploy, configure, and tune the matching pipeline.
• Solution architects — design the contextualization approach across source systems.

• You need to match time series (or another entity view) to assets automatically, without building a custom matching pipeline from scratch.
• Your project combines data from multiple sources (for example, SAP assets and OSIsoft/AVEVA PI time series) and requires relationship discovery across them.
• You need a combination of manual expert mappings, rule-based logic, and AI-powered matching in a single orchestrated workflow.
• You need incremental processing — only new or updated entities should be matched on each run.
• You want a scalable foundation proven for 10,000+ time series per batch in production environments.

• If your assets and time series are already contextualized and you only need to maintain relationships — this pack is optimized for initial and incremental matching, not purely for maintenance.
• If your project requires document or P&ID annotation, use the P&ID Annotation deployment pack instead.

​

What’s included

​

Prerequisites

• Cognite Toolkit version 0.8 or later is installed. See Setting up.
• A cdf.toml file exists in your project root. If it is missing, run cdf modules init and select Create toml file (required).
• The data plugin is enabled in cdf.toml — required for the cdf data upload step after deployment:
  [plugins]
  data = true
  
• Authentication is configured and verified — run cdf auth init and cdf auth verify using a local .env file. See Authentication and authorization.
• A data model with the target view (for example, Asset) and entity view (for example, TimeSeries) is deployed, and instances are already ingested.
• You have a service principal (app registration) in your identity provider (IdP) for the scheduled workflow to authenticate as. Have its client ID and client secret ready — these populate IDP_CLIENT_ID / IDP_CLIENT_SECRET in your .env (mapped to workflowClientId / workflowClientSecret).
• You have an IdP group for the processing access group, and you know its source ID — the group’s object ID in the IdP (populates GROUP_SOURCE_ID, mapped to entity_matching_processing_group_source_id). The workflow’s service principal must be a member of this group.

​

Install and deploy

cdf modules add -d cdf_entity_matching

cdf build

cdf deploy --dry-run

cdf deploy

cdf data upload dir ./modules/contextualization/cdf_entity_matching/upload_data/ -v

modules/
└── contextualization/
    └── cdf_entity_matching/

​

Verify the deployment

1. The cdf deploy command finishes without errors and reports successful deployment of Functions, Workflows, Extraction Pipelines, RAW, the data set, the access group, and the data modeling resources.
2. The cdf data upload command finishes without errors, and contextualization_manual_input and contextualization_rule_input are visible in Integrate > Staging (RAW) under db_asset_entity_matching.
3. In CDF, navigate to Integrate > Data Workflows and verify that the EntityMatching workflow is listed.
4. Trigger the workflow with Run (or wait for the daily EntityMatching_trigger). Confirm that the metadata update task completes before the entity matching task, and that both finish without errors.
5. Navigate to Build Solutions > Functions, open fn_dm_context_metadata_update and fn_dm_context_timeseries_entity_matching, select View logs, and review the metrics (processed/updated counts, cache hit rate, match scores).
6. Navigate to Integrate > Staging (RAW) and verify that contextualization_good and contextualization_bad are populated.
7. Use CDF Search or Data Modeling to confirm that direct relationships exist between matched entities and the target view (when dmUpdate is enabled).

​

Configuration reference

​

Runtime parameters (extraction pipelines)

​

Architecture

​

Data flow

​

Matching pipeline

1. Read configuration and the RAW input tables (manual mappings, rule mappings).
2. Apply manual mappings from entity to target — these overwrite any existing mapping.
3. Read the entities not yet matched (or all entities when runAll is true).
4. Read the target view instances (for example, assets).
5. Run rule-based matching using the configured regex patterns.
6. Run ML entity matching in CDF for the remaining candidates.
7. Update entity→target relationships in the data model when dmUpdate is true.
8. Write results to contextualization_good (score ≥ autoApprovalThreshold) and contextualization_bad (below threshold).

​

Incremental processing and state management

​

Resilience

​

Troubleshooting

​

Support