Skip to main content
Use this page when several people change the Cognite Toolkit configuration for one Cognite Data Fusion (CDF) project. You will include shared configuration files, build several config files in one command, deploy only the modules Git shows as changed, and stop two people from deploying the same module at the same time.
These steps use alpha flags. The flags are experimental and may change or be removed without notice. Enable them on a shared development project. For staging and production, deploy with your continuous integration and continuous delivery (CI/CD) pipeline.
Before you begin, ensure you have:
  • Installed the Cognite Toolkit 0.9.9 or later for multiple config files, 0.9.21 or later for git diff selection, 0.9.26 or later for deployment protection, and 0.9.70 or later for configuration file inclusion. Version details are on Alpha flags.
  • Completed Setting up the Cognite Toolkit locally.
  • Initialized a project repository with a cdf.toml file and environment configuration files.
  • Confirmed access to a Git repository and a CI/CD pipeline.

Understanding collaboration challenges

When you configure CDF projects across multiple sites, several team members often change configuration at the same time. Projects also span development, testing, staging, and production. Uncoordinated edits then overwrite each other’s work during merges and deployments, and resource definitions drift when people skip configuration best practices. This page shows how to prevent those conflicting changes. For inconsistent configurations, use the Cognite Toolkit validation rules. To promote a configuration from development to production, follow Setting up CI/CD for the Cognite Toolkit.

Setting up the team

To collaborate effectively when managing CDF configurations with the Cognite Toolkit, follow these recommended practices: Use version control. Manage all configuration files in a version control system such as Git. This tracks configuration history and allows you to revert changes when needed. Use a monorepo. Store all module and environment configurations in a single repository. A monorepo provides a single source of truth and simplifies dependency management between resources. Protect default branches. Enable branch protection on default branches (such as main). Require team members to propose changes through pull requests to ensure code review and automated validation before merging. Automate with CI/CD. Implement a CI/CD pipeline to validate and deploy configuration changes automatically. Run automated checks on every pull request, and use a deployment pipeline on the default branch to publish changes to the target CDF environment.

Editing modules

Follow this workflow when modifying configuration modules:
1

Create a feature branch

Create a branch from the default branch (main). Use a descriptive branch name that follows your team’s naming conventions.
2

Edit the module

Edit only the module files relevant to your change in your local branch.
3

Create a pull request

Open a pull request targeting the default branch. Include a clear description of the proposed configuration changes.
4

Run automated checks

Verify that all automated CI/CD checks pass, including cdf build validation.
5

Request a review

Request a review from your team or designated module owners.
6

Apply requested changes

If reviewers request changes, commit and push updates to your feature branch. The pull request updates automatically.
7

Merge the pull request

After receiving approvals and passing all checks, merge the pull request into the default branch.
8

Deploy changes

The CI/CD pipeline deploys the merged configuration to the target CDF environment.Confirm that the pipeline run succeeds and that the resources in the target CDF project match the merged module files.

Building multiple config files

Teams often maintain a single config.[env].yaml file per environment, such as config.dev.yaml. When multiple team members edit the variables section at the same time, merge conflicts can occur. You can split that configuration into site-specific files, such as config.site1-dev.yaml and config.site2-dev.yaml. Building those files in separate commands can produce conflicting resource definitions. Build them in one cdf build command so the Cognite Toolkit checks that the resources agree. Each file keeps its own environment.selected and variables. To merge files into one configuration, see Including configuration files.
1

Enable multiple config files

Set the multi-configs alpha flag in cdf.toml:
cdf.toml
2

Build the config files together

Pass each file to cdf build. Use the same environment.project and the same environment.validation-type in every file. environment.selected and variables stay independent, so each file can build the same modules with different variable values.
3

Confirm the build

cdf build completes. The Cognite Toolkit validates the resources from every config file in that command. When environment.project or environment.validation-type differs between the files, align those fields and run the command again.

Including configuration files

Use include when several configuration files should become one configuration for a single cdf build. Team members can keep domain-specific files separate and share variables across those files. To keep each file separate, with its own environment.selected and variables, build multiple config files in one command. With include, environment.selected is the union of every file, and the same variable key path in more than one file is an error. When you enable config-include, the Cognite Toolkit merges the included files into the config.[env].yaml file that lists them:
  1. environment.selected is the union of the modules listed in selected across all files.
  2. Any other environment field must match the main file wherever an included file sets it. An included file can omit environment when it only defines variables.
  3. The Cognite Toolkit merges variables mappings. Nested mappings combine, so each file can add its own keys. Setting the same key path in more than one file is an error. For example, modules.core.sourceId in two files stops the build. modules.core.sourceId in one file and modules.site1.sourceId in another file merge.
1

Enable configuration inclusion

Set the config-include alpha flag in cdf.toml:
cdf.toml
2

Add include paths

Add include to your config.[env].yaml file. Each entry is a path to another configuration YAML file:
config.dev.yaml
Each path is relative to the directory of the file that lists it. The path must be a .yaml or .yml file in that directory or a subdirectory. An included file can list further files in its own include field. The Cognite Toolkit loads each file once.
3

Define the included configuration files

Omit environment in a file that only sets variables:
config.shared.yaml
In a file that adds modules, set selected. Leave the other environment fields in the main file:
config.site1.yaml
If you set another environment field in an included file, it must equal the same field in the main file.
4

Build the configuration

Run cdf build with your main environment file:
The Cognite Toolkit merges the included files, combines environment.selected, and validates the resources. If the same variable key path is set in more than one file, or an environment field in an included file does not match the main file, the command stops with an error.

Selecting changed modules with git diff

Git diff selection builds and deploys only the intersection of modules specified in environment.selected and modules whose YAML files changed relative to deploy-branch. Enable it with the git-diff-select alpha flag, then set selection-type: git-diff. Use git diff selection on a shared development project. In staging and production, deploy every configured module through your CI/CD pipeline. Selecting every module on each deploy can overwrite work that other people still have on feature branches. When environment.selected lists all modules, every deploy builds and publishes all of them:
config.dev.yaml
Commenting modules out of environment.selected by hand is easy to get wrong:
config.dev.yaml
With git diff selection, you keep environment.selected configured without manually commenting modules in and out. The Cognite Toolkit requires environment.selected to be set and only builds modules that are both listed in environment.selected and modified according to Git history.
1

Enable git diff selection

Set the git-diff-select alpha flag in cdf.toml:
cdf.toml
2

Configure selection and the deploy branch

In the environment configuration file, set selection-type to git-diff, set validation-type to dev, set deploy-branch to the branch you deploy from, and configure selected. The Cognite Toolkit requires environment.selected to be set and rejects selection-type: git-diff when validation-type is anything other than dev.
config.dev.yaml
3

Commit the module YAML

Commit the changed .yaml and .yml files under modules/ before you build. Git diff selection reads committed files. When those files are still uncommitted, cdf build stops and lists them.
4

Build and deploy the changed modules

Run cdf build, then cdf deploy.On a feature branch, the Cognite Toolkit runs git diff against deploy-branch and builds only the modules that are in environment.selected and differ from deploy-branch.On deploy-branch, the Cognite Toolkit runs git diff against the previous commit. Merge pull requests into that branch with Squash and merge. Otherwise, that diff cannot detect the full set of changed modules.
5

Confirm the selection

Run cdf build --verbose. The Selection section lists only the module paths that are in environment.selected and differ from deploy-branch.When no changed module YAML matches environment.selected, the command stops with No modules were found to build based on the git diff selection.

Protecting deployments from concurrent overwrites

Deployment protection stops two people from deploying the same module to a shared development project at the same time. Enable it with the deploy-protection alpha flag, then set deploy-protection: protected. You can use it with or without git diff selection. In staging and production, run deployments through your CI/CD pipeline. Deployment protection is for shared development projects. It leaves access control unchanged, and a person who already holds the lock can deploy again from that same branch.
1

Enable deployment protection

Set the deploy-protection alpha flag in cdf.toml:
cdf.toml
2

Protect the development environment

In the environment configuration file, set deploy-protection to protected and set deploy-branch.
config.dev.yaml
To combine deployment protection with git diff selection, add selection-type and selected to that same environment block:
config.dev.yaml
3

Deploy from your branch

Run cdf deploy from a Git repository.On a feature branch, the Cognite Toolkit requests a lock for each module in the deployment. It grants the lock when the module is unlocked, the lock is already on your current branch, or the lock is more than 24 hours old. When another branch holds a current lock, cdf deploy stops and deploys nothing.On deploy-branch, cdf deploy takes the locks for the modules in the deployment and releases them when the deployment finishes.
4

Confirm the locks

A successful deploy prints a panel titled Successfully acquired locks. followed by the module identifiers.When another branch holds a lock, the command stops with Failed to acquire locks for modules: and names the module, the Git user who holds the lock, and that user’s branch.After a successful deploy on deploy-branch, the Cognite Toolkit prints a panel titled Successfully released locks.

Further reading

Last modified on November 11, 2026