Skip to main content
The Atlas AI Property Extractor (dp:atlas_ai:ai_extractor) reads free text from a source property (for example, a maintenance notification description) and fills structured target properties (for example, discipline, priority, category) using an LLM agent deployed through Cognite Atlas AI. It deploys as a Cognite Toolkit module and runs a scheduled CDF Function that turns unstructured documentation into queryable data model properties without manual entry. This module writes into an existing CDF view and supports per-property write modes — add_new_only, append, and overwrite — with epoch-based incremental processing.

​
Who it’s for

Target personas:
  • Data engineers — deploy and configure the extractor and its schedule.
  • Subject matter experts — domain experts who choose which properties to extract and validate results.
Use this pack when:
  • You have documents, notifications, or free-text fields in CDF that contain technical properties (discipline, priority, category, tags) you want extracted and stored as structured data model properties.
  • You want to enrich an existing CDF view with AI-generated summaries or derived metadata without building a custom extraction pipeline.
  • You need to accumulate tags or keywords from multiple sources over time (append mode).
  • You are building agentic solutions and need a structured data foundation for agentic reasoning and industrial search.
When not to use this pack:
  • If your CDF project does not have Atlas AI enabled — this module requires an Atlas AI agent and cannot run without it.
  • If you do not yet have a target CDF view — the module writes into an existing view and will not create one. Deploy your data model first.
  • If you need rule-based or SQL-driven matching rather than LLM extraction — use the CDF Entity Matching pack instead.

​
What’s included

​
Prerequisites

Verify all of the following before you start:
  • Cognite Toolkit version 0.8 or later is installed. See Setting up.
  • A cdf.toml file exists in your project root. If it is missing, run cdf modules init and select Create toml file (required).
  • Authentication is configured and verified — run cdf auth init and cdf auth verify using a local .env file. See Authentication and authorization.
  • Your CDF project has Atlas AI Agents enabled.
  • A target CDF view already exists with a text property to read from and the properties you want to populate. For append/overwrite, the view also needs a Timestamp property for aiTimestampProperty.
  • You have a service principal (app registration) in your identity provider (IdP) for the scheduled workflow to authenticate as. Have its client ID and client secret ready — these populate IDP_CLIENT_ID / IDP_CLIENT_SECRET in your .env (mapped to workflowClientId / workflowClientSecret).

​
Install and deploy

You can add the module to an existing Toolkit project or start a new project from scratch.
The AI Property Extractor is the cdf_ai_extractor module of the Atlas AI deployment pack (dp:atlas_ai), which also ships out-of-the-box agents. Add this module from your project directory:
The Toolkit creates the module under modules/solutions/cdf_ai_extractor/ and updates your config.<env>.yaml files. Before building, open config.dev.yaml and set at minimum the source view (viewSpace, viewExternalId, viewVersion), the textProperty, the properties to extract, the credentials (workflowClientId / workflowClientSecret), and — if using append/overwrite — the aiTimestampProperty. See Configuration reference. Then build and deploy:
1

Build deployment artifacts

The Toolkit substitutes template variables and writes artifacts to the build/ directory. If you need to change the project name or any variables marked <change_me>, edit config.<env>.yaml first — see Configure, build, and deploy modules.
2

Dry-run the deployment

Inspect the output and confirm that configurations look correct before deploying.
3

Deploy to CDF

The Toolkit deploys only configurations that have changed since the last run.
4

Set up CI/CD (optional)

For governed production deployments, automate build, dry-run, and deploy in a pipeline. See Set up CI/CD pipelines.
After deployment, your project should include:

​
Verify the deployment

After deploying, confirm success with these checks:
  1. The cdf deploy command finishes without errors and reports successful deployment of the agent, function, workflow, extraction pipeline, dataset, and RAW resources.
  2. In CDF, navigate to Build Solutions > Functions and confirm fn_ai_property_extractor is listed with a Ready status.
  3. Navigate to Integrate > Data Workflows and confirm wf_ai_property_extractor is registered and its schedule is active.
  4. Trigger the workflow (or call the function directly with logLevel: DEBUG) to run an initial extraction, and monitor it to completion.
  5. Open the function logs and confirm that instances were processed and properties were written without errors.
  6. Open the target view in Data Modeling and verify that the extracted properties appear on the expected instances.
  7. In Integrate > Extraction pipelines, confirm ep_ai_property_extractor reports a successful run.
The deployment is successful when the function is ready, the workflow runs without errors, and extracted properties appear in the target view.

​
Configuration reference

Module variables are set per environment in config.<env>.yaml. Update these before running cdf build. Keep client IDs and secrets in your .env file, not hardcoded. Each variable takes its default unless you override it. Where the default is —, a placeholder, or a ${...} value read from .env, you must provide your own value before deploying.

​
Write modes

Each property can have its own write mode, configured in the extraction pipeline config’s per-property properties list.

​
Architecture

The module deploys an Atlas AI agent, a CDF Function, a scheduled workflow, an extraction pipeline holding the runtime config, a RAW state store, and a dataset.

​
Data flow

On each scheduled or manual run, the function loops — querying a batch, calling the agent, writing results, and stamping the timestamp — until all matching instances are processed or ~9 minutes elapse. The state store lets the next run resume where the previous one stopped.

​
Extraction loop

  1. Query a batch — select instances from the source view where the text property exists and, depending on write mode, the target is empty (add_new_only) or the AI timestamp predates the current epoch (append / overwrite).
  2. Invoke the agent — send the text to ai_property_extractor_agent, which returns the extracted property values (optionally several instances per call via llmBatchSize).
  3. Write to the target view — upsert the values into the target view, following each property’s write mode.
  4. Stamp and repeat — set aiTimestampProperty on processed instances (append/overwrite), update the state cursor, and continue until done or the time budget is reached.

​
State and re-runs

State is a single RAW row in ai_extractor_state/extraction_state holding epoch_start, cursor, and config_version. To reprocess everything, call the function with resetState: true, bump stateStoreConfigVersion, or disable the state store to process all matching instances every run. Manual runs accept ExtractionPipelineExtId, logLevel, and resetState as input parameters.

​
Troubleshooting

For issues not covered here, contact Cognite support.

​
Support

Last modified on July 6, 2026