Skip to main content
The Atlas AI Property Extractor (dp:atlas_ai:ai_extractor) reads free text from a source property (for example, a maintenance notification description) and fills structured target properties (for example, discipline, priority, category) using an LLM agent deployed through Cognite Atlas AI. It deploys as a Cognite Toolkit module and runs a scheduled CDF Function that turns unstructured documentation into queryable data model properties without manual entry. This module writes into an existing CDF view and supports per-property write modes — add_new_only, append, and overwrite — with epoch-based incremental processing.

​
Who it’s for

Target personas:
  • Data engineers — deploy and configure the extractor and its schedule.
  • Subject matter experts — domain experts who choose which properties to extract and validate results.
Use this pack when:
  • You have documents, notifications, or free-text fields in CDF that contain technical properties (discipline, priority, category, tags) you want extracted and stored as structured data model properties.
  • You want to enrich an existing CDF view with AI-generated summaries or derived metadata without building a custom extraction pipeline.
  • You need to accumulate tags or keywords from multiple sources over time (append mode).
  • You are building agentic solutions and need a structured data foundation for agentic reasoning and industrial search.
When not to use this pack:
  • If your CDF project does not have Atlas AI enabled — this module requires an Atlas AI agent and cannot run without it.
  • If you do not yet have a target CDF view — the module writes into an existing view and will not create one. Deploy your data model first.
  • If you need rule-based or SQL-driven matching rather than LLM extraction — use the CDF Entity Matching pack instead.

​
What’s included

ComponentDescriptionOptional?
Atlas AI agent (ai_property_extractor_agent)LLM agent that reads the source text and returns the extracted property values.No
Extractor function (fn_ai_property_extractor)CDF Function that queries source instances, calls the agent, writes results to the target view, and stamps the AI timestamp. Runs in a loop up to ~9 minutes per invocation.No
Workflow (wf_ai_property_extractor)Orchestrates the function, with a scheduled trigger (default daily at 04:00 UTC). Can also be triggered manually.No
Extraction pipeline (ep_ai_property_extractor)Holds the runtime configuration (views, properties, write modes, batching, prompt, state store) and tracks run status.No
RAW state storeai_extractor_state database with the extraction_state table — tracks the processing epoch and cursor for incremental runs.No
Dataset (ai_extractor)CDF dataset for lineage tracking.No

​
Prerequisites

Verify all of the following before you start:
  • Cognite Toolkit version 0.8 or later is installed. See Setting up.
  • A cdf.toml file exists in your project root. If it is missing, run cdf modules init and select Create toml file (required).
  • Authentication is configured and verified — run cdf auth init and cdf auth verify using a local .env file. See Authentication and authorization.
  • Your CDF project has Atlas AI Agents enabled.
  • A target CDF view already exists with a text property to read from and the properties you want to populate. For append/overwrite, the view also needs a Timestamp property for aiTimestampProperty.
  • You have a service principal (app registration) in your identity provider (IdP) for the scheduled workflow to authenticate as. Have its client ID and client secret ready — these populate IDP_CLIENT_ID / IDP_CLIENT_SECRET in your .env (mapped to workflowClientId / workflowClientSecret).

​
Install and deploy

You can add the module to an existing Toolkit project or start a new project from scratch.
The AI Property Extractor is the cdf_ai_extractor module of the Atlas AI deployment pack (dp:atlas_ai), which also ships out-of-the-box agents. Add this module from your project directory:
cdf modules add -d cdf_ai_extractor
The Toolkit creates the module under modules/solutions/cdf_ai_extractor/ and updates your config.<env>.yaml files. Before building, open config.dev.yaml and set at minimum the source view (viewSpace, viewExternalId, viewVersion), the textProperty, the properties to extract, the credentials (workflowClientId / workflowClientSecret), and — if using append/overwrite — the aiTimestampProperty. See Configuration reference. Then build and deploy:
1

Build deployment artifacts

cdf build
The Toolkit substitutes template variables and writes artifacts to the build/ directory. If you need to change the project name or any variables marked <change_me>, edit config.<env>.yaml first — see Configure, build, and deploy modules.
2

Dry-run the deployment

cdf deploy --dry-run
Inspect the output and confirm that configurations look correct before deploying.
3

Deploy to CDF

cdf deploy
The Toolkit deploys only configurations that have changed since the last run.
4

Set up CI/CD (optional)

For governed production deployments, automate build, dry-run, and deploy in a pipeline. See Set up CI/CD pipelines.
After deployment, your project should include: 
modules/
└── solutions/
    └── cdf_ai_extractor/

​
Verify the deployment

After deploying, confirm success with these checks:
  1. The cdf deploy command finishes without errors and reports successful deployment of the agent, function, workflow, extraction pipeline, dataset, and RAW resources.
  2. In CDF, navigate to Build Solutions > Functions and confirm fn_ai_property_extractor is listed with a Ready status.
  3. Navigate to Integrate > Data Workflows and confirm wf_ai_property_extractor is registered and its schedule is active.
  4. Trigger the workflow (or call the function directly with logLevel: DEBUG) to run an initial extraction, and monitor it to completion.
  5. Open the function logs and confirm that instances were processed and properties were written without errors.
  6. Open the target view in Data Modeling and verify that the extracted properties appear on the expected instances.
  7. In Integrate > Extraction pipelines, confirm ep_ai_property_extractor reports a successful run.
The deployment is successful when the function is ready, the workflow runs without errors, and extracted properties appear in the target view.

​
Configuration reference

Module variables are set per environment in config.<env>.yaml. Update these before running cdf build. Keep client IDs and secrets in your .env file, not hardcoded. Each variable takes its default unless you override it. Where the default is —, a placeholder, or a ${...} value read from .env, you must provide your own value before deploying.
VariableDescriptionDefault
projectCDF project external ID to deploy to.—
viewSpace / viewExternalId / viewVersionSource view containing the instances to read from.cdf_cdm / CogniteAsset / v1
textPropertyProperty on the source view containing the text to extract from.description
propertiesToExtractJSON array of property IDs to extract. Empty extracts all non-filled properties.[]
targetViewSpace / targetViewExternalId / targetViewVersionTarget view for writing, if different from the source view.— / — / v1
aiTimestampPropertyTimestamp property on the target view used to track processed instances. Required for append / overwrite.""
aiPropertyMappingJSON object mapping source properties to different target properties.{}
processingFiltersJSON array of extra data model filters for instance selection.[]
customPromptInstructionsExtra instructions appended to the LLM prompt.""
batchSizeInstances per query batch (1–100).10
llmBatchSizeInstances per LLM call (1–50).1
functionSpaceSpace for the function deployment.—
agentExternalIdAtlas AI agent external ID.ai_property_extractor_agent
extractionPipelineExternalIdExtraction pipeline external ID.ep_ai_property_extractor
stateStoreEnabled / stateStoreDatabase / stateStoreTableState tracking toggle and RAW location.true / ai_extractor_state / extraction_state
stateStoreConfigVersionBump to trigger a full re-run (new epoch).v1
workflowWorkflow external ID.wf_ai_property_extractor
scheduleExpressionCron schedule for the workflow.0 4 * * *
workflowClientId / workflowClientSecretIdP credentials for the scheduled workflow.${IDP_CLIENT_ID} / ${IDP_CLIENT_SECRET}

​
Write modes

Each property can have its own write mode, configured in the extraction pipeline config’s per-property properties list.
ModeBehaviorAI timestamp required?Use case
add_new_only (default)Writes only when the target property is empty.NoOne-time enrichment.
appendMerges new values into existing lists and deduplicates.YesAccumulating tags over time.
overwriteAlways replaces the target value with the new extraction.YesRe-extraction after an LLM upgrade.

​
Architecture

The module deploys an Atlas AI agent, a CDF Function, a scheduled workflow, an extraction pipeline holding the runtime config, a RAW state store, and a dataset.

​
Data flow

On each scheduled or manual run, the function loops — querying a batch, calling the agent, writing results, and stamping the timestamp — until all matching instances are processed or ~9 minutes elapse. The state store lets the next run resume where the previous one stopped.

​
Extraction loop

  1. Query a batch — select instances from the source view where the text property exists and, depending on write mode, the target is empty (add_new_only) or the AI timestamp predates the current epoch (append / overwrite).
  2. Invoke the agent — send the text to ai_property_extractor_agent, which returns the extracted property values (optionally several instances per call via llmBatchSize).
  3. Write to the target view — upsert the values into the target view, following each property’s write mode.
  4. Stamp and repeat — set aiTimestampProperty on processed instances (append/overwrite), update the state cursor, and continue until done or the time budget is reached.

​
State and re-runs

State is a single RAW row in ai_extractor_state/extraction_state holding epoch_start, cursor, and config_version. To reprocess everything, call the function with resetState: true, bump stateStoreConfigVersion, or disable the state store to process all matching instances every run. Manual runs accept ExtractionPipelineExtId, logLevel, and resetState as input parameters.

​
Troubleshooting

SymptomLikely causeResolution
Function shows a failed status after deploymentAtlas AI Agents not enabled, or the Toolkit account is missing capabilities.Confirm Atlas AI Agents are enabled, and that functions:write and workflows:write are granted to the Toolkit CDF group.
"Agent not found"The ai_property_extractor_agent agent is not deployed or accessible.Verify the agent deployed successfully and the project has Atlas AI enabled.
"View not found"viewSpace / viewExternalId / viewVersion are incorrect.Confirm the external IDs in config.<env>.yaml match a view in your project.
Workflow runs but no properties are writtenSource/target view misconfigured, text property empty, or targets already filled (add_new_only).Check the function logs; confirm the views, textProperty, and target properties, and that instances have text to read.
Same records re-extracted on every runaiTimestampProperty is missing or does not match a Timestamp property on the target view.Add aiTimestampProperty to the config and create it as a Timestamp property on the target view.
"aiTimestampProperty is required"A property uses append or overwrite without a configured timestamp.Configure aiTimestampProperty and add the Timestamp property to the target view.
Values never refreshed after an LLM upgradeRunning in add_new_only/append when a full refresh is needed.Use overwrite for the affected properties, or trigger a full re-run with resetState: true or by bumping stateStoreConfigVersion.
Workflow schedule never firesInvalid cron (for example, February 29 in a non-leap year).Update scheduleExpression to a valid recurring cron expression.
Function timeoutBatches too large or prompt too heavy.Reduce batchSize, simplify customPromptInstructions, or raise llmBatchSize.
For issues not covered here, contact Cognite support.

​
Support

FieldDetails
TeamCognite Value Delivery Team
DocumentationCognite product documentation
Questions and feedbackDeployment Packs group on Cognite Hub
Version1.0.0
Last updated2026
Last modified on July 6, 2026