Skip to main content
This guide helps you set up the Cognite Toolkit in Azure DevOps Pipelines to automate the deployment of your modules with version control and continuous integration. Prerequisites:
  • Access to an Azure DevOps project with pipeline creation permissions
  • A Cognite Data Fusion (CDF) project with authentication configured for the Cognite Toolkit
  • The Cognite Toolkit installed and configured locally
  • A Git repository connected to Azure DevOps
You’ll set up two pipelines: one for validating pull requests (CI) and one for automatic deployments (CD).
When you promote across dev, test, and prod, use the Azure DevOps pipelines that the Foundation deployment pack generates. It creates one pipeline per environment, validates pull requests with cdf build (no credentials), and deploys production from an Azure DevOps tag vX.Y.Z.This guide uses cdf repo init, which writes starter dry-run and deploy pipelines for one environment. If the Foundation pack is installed, run python modules/common/cdf_project_foundation/scripts/generate_actions.py --provider ado instead. Follow the generated docs/FOUNDATION_CICD.md for file names and variable groups, then complete Enable pull request validation (CI) and Set up automatic deployment (CD).

​
Add pipelines to your repository

1

Generate pipeline configuration files

In a terminal, run these commands and select Azure DevOps as the CI/CD provider to create a folder with example pipelines:
The command writes the example pipelines to .devops. Adapt the generated files to your branches and CDF environments. Pin the Cognite Toolkit version to [modules].version in cdf.toml.
2

Customize and commit the pipelines

Adapt the pipelines to your needs, and then commit the changes to a new branch and push it to your repository:
3

Create a pull request

Create a pull request to merge the add-ado-pipelines branch into main.
After merging, the .devops folder with pipeline configuration files is available in your main branch.

​
Enable pull request validation (CI)

Automatic validation ensures that modules are valid before merging pull requests into the main branch.
1

Create a dry-run pipeline

  1. In your DevOps project, navigate to Pipelines.
  2. Select New pipeline.
  3. Select the repository you pushed the changes to.
  4. Select Existing Azure Pipelines YAML file and specify:
    • Branch: main
    • Path: ./.devops/dry-run-pipeline.yml
Add new pipeline in Azure DevOps
  1. Select Save (alternative to the Run button).
  2. In the pipeline overview, click the three dots and select Rename/move to set a descriptive name, for example “Pull request checks”.
2

Create variable groups for the dev environment

  1. In your DevOps project, navigate to Pipelines > Library.
  2. Select + Variable group.
  3. Create a variable group with the name dev-toolkit-credentials.
  4. Select Pipeline permissions and grant access to the pipelines that should use these credentials.
  5. Add these variables with the correct values for your environment:
    • CDF_CLUSTER — your CDF cluster (for example, westeurope-1)
    • CDF_PROJECT — your CDF project name
    • LOGIN_FLOW — set to client_credentials
    • IDP_CLIENT_ID — your application client ID
    • IDP_TENANT_ID — your Microsoft Entra ID tenant ID
    • IDP_CLIENT_SECRET — your client secret (important: mark as secret using the padlock icon)
    • IDP_TOKEN_URL — only if you are not using Microsoft Entra ID
For Microsoft Entra ID, IDP_TOKEN_URL is configured automatically. Set it only if you use a different identity provider.
3

Add the pipeline to branch policy

  1. In your DevOps project, navigate to Repos > Branches.
  2. Click the three dots next to the main branch and select Branch policies.
  3. Under Build Validation, click the + button.
  4. Select the dry-run pipeline you created.
  5. Set the minimum number of reviewers to 1.
All new pull requests now require a successful build and dry-run before merging.

​
Set up automatic deployment (CD)

Automatically deploy modules when changes are pushed to the main branch.
1

Create a deployment pipeline

  1. In your DevOps project, navigate to Pipelines.
  2. Select New pipeline.
  3. Select your repository.
  4. Select Existing Azure Pipelines YAML file and specify:
    • Branch: main
    • Path: ./.devops/deploy-pipeline.yml
  5. Select Save (alternative to the Run button).
2

Create variable groups for the target environment

  1. In your DevOps project, navigate to Pipelines > Library.
  2. Select + Variable group.
  3. Create a variable group with the name <environment>-toolkit-credentials (e.g., prod-toolkit-credentials for production).
  4. Select Pipeline permissions and grant access to the deployment pipeline.
  5. Add these variables with the correct values for your target environment:
    • CDF_CLUSTER — your CDF cluster
    • CDF_PROJECT — your CDF project name
    • LOGIN_FLOW — set to client_credentials
    • IDP_CLIENT_ID — your application client ID
    • IDP_TENANT_ID — your Microsoft Entra ID tenant ID
    • IDP_CLIENT_SECRET — your client secret (important: mark as secret using the padlock icon)
    • IDP_TOKEN_URL — only if you are not using Microsoft Entra ID
Use separate variable groups and credentials for each environment (dev, test, prod) to maintain proper security boundaries.
The pipeline now automatically deploys modules when changes are pushed to the main branch.
Successful deployment job in Azure DevOps

​
Troubleshooting

​
OAuth 2 MUST utilize https error

Error message:
Solution: This error indicates incorrectly configured environment variables. Verify that all environment variables are set correctly in your variable group, particularly the IDP_TOKEN_URL which must use HTTPS.
Last modified on September 15, 2026