Skip to main content
Treat contextualization as a repeatable pipeline, not a one-off project. After you prepare aliases and match fields, you scope which sources and targets a run can see, apply deterministic links before machine learning, store what did not match, and rerun when data or mappings change. This procedure is for data engineers who run matching in Cognite Data Fusion (CDF) or through the CDF Entity Matching and File Annotation deployment packs. It does not replace the CDF Entity Matching configuration reference, File Annotation configuration reference, or deployment pack troubleshooting tables. Use those pages for YAML flags and error messages.

​
Prerequisites

Before you start:
  • Complete alias and match-field preparation on enterprise targets. See Prepare aliases and match fields.
  • Functional locations and sources live in the enterprise layer with stable identities. Functional locations are assets in the graph. See About contextualization.
  • Decide whether you run matching in the CDF UI (Match entities) or a Cognite Toolkit workflow.
Do not bulk-delete subject matter expert (SME)-confirmed matches, manual mapping tables, or regex rules to “start fresh.” Those artifacts are production data. Scope a rerun, adjust aliases, or reprocess only the entities that changed.

​
Run and maintain pipelines

1

Scope candidates by site, plant, or unit

Limit each run by site, plant, or unit. Identical tags at different sites must not match each other. Aliases make the same tag text match across sites, and scope stops those false matches. Pick one scoping mechanism and use it consistently.
A location filter scopes search and 3D in apps. It does not replace instance-space or data set scoping for matching pipelines.
2

Apply deterministic links before machine learning

Run the simplest approach that can create the relationship:
  1. Exact business key. Write the link in a transformation when source and target share a stable key.
  2. Manual expert mappings. Load curated source-to-target pairs (the CDF Entity Matching deployment pack stores these in contextualization_manual_input).
  3. Regex rules. Encode repeatable patterns (the CDF Entity Matching deployment pack stores these in contextualization_rule_input; the CDF UI can save generated rules with a pipeline).
  4. Machine learning. Use entity matching for the remainder.
The CDF Entity Matching deployment pack runs manual mappings, then rules, then ML. See Architecture in the deployment pack reference. Prefer adding a rule or mapping over retraining when the failure mode is a known pattern.
3

Store and review unresolved matches

Not every run produces a link. Plan for review instead of treating low coverage as a model-only problem.
  • In the CDF UI, filter for Unmatched entities or Different recommendation when you validate a pipeline. See Match entities.
  • In the CDF Entity Matching deployment pack, scores below autoApprovalThreshold land in contextualization_bad; approved matches land in contextualization_good. Review bad rows for missing targets, bad aliases, or scope that is too narrow or too wide.
  • For diagram annotation, inspect files that returned no tag links before you rerun OCR or parsing. Low recall often means voided tags or missing aliases on assets, not a weak parser. If aliases look correct, see Troubleshooting interactive diagrams for parser configuration.
Feed accepted fixes back into Prepare aliases and match fields or manual mapping tables. The next run must not repeat the same miss.
4

Configure incremental reruns

After the first pass, process only what changed unless you have a documented reason to reprocess everything.CDF UI
  • Save a pipeline tied to data sets. New ingested entities then enter the same model. When a data set receives new data, select Rerun pipeline on the entity matching overview. See Rerun a matching model.
  • Confirmed matches and saved rules persist with the pipeline. Reruns apply to new or unmatched entities according to the filters you selected.
Cognite Toolkit deployment packs
  • Set runAll: false. The Entity Matching function then processes only entities updated since the last run. State is stored in contextualization_state_store. See Incremental processing and state management.
  • For File Annotation, keep cleanOldAnnotations false on incremental runs so annotations from earlier runs stay. Set it to true only when you need to replace those annotations. See Matching.
Shipped deployment pack defaults often use runAll: true for first deployment. Switch to incremental mode after you validate the initial match rate.
5

Decide when a full reprocess is justified

Reprocess all entities or files only when the change affects matching logic globally, for example:
  • You replaced the enterprise asset hierarchy or moved instance spaces.
  • You changed targetViewSearchProperty, entityViewSearchProperty, or tokenizer-related fields across the project.
  • You merged rename lists into aliases on a large share of targets and need to re-evaluate prior ML suggestions.
For a full reprocess in Cognite Toolkit deployment packs, set runAll: true or clear the relevant state table, then run the workflow. In the CDF UI, open the pipeline, adjust scope or model settings, and rerun on the full data set.If you change autoApprovalThreshold in the CDF Entity Matching deployment pack, clean up relationships created by previous automated runs before you re-run. The deployment pack reference notes that manually created relationships are preserved. See Runtime parameters.
6

Trigger reruns when inputs change

Schedule or trigger pipelines when any of these change:Use CDF Workflows or scheduled triggers to run Cognite Toolkit modules after upstream transforms finish ingesting data.

​
UI and Cognite Toolkit at a glance

If you are deciding where to run matching day to day, use this table to compare the CDF UI with Cognite Toolkit deployment packs before you return to the steps above for scoping and rerun rules.

​
Next steps

Last modified on October 28, 2026