Setting up the Cognite Toolkit

Follow the steps in this article to install the Cognite Toolkit and the basic file structure on your computer and prepare the configuration for integration with version control repository.

We highly recommend that you use Git and a service like GitHub or Azure DevOps to manage the project configurations "as code". For governance of production projects, we also recommend using a CI/CD pipeline to deploy changes to Cognite Data Fusion.

Prerequisites

• A recent version of Python, at minimum v3.10.
• Recommended: a virtual environment manager like Poetry or pyenv for Python versions and dependencies.
• Recommended: Basic knowledge of Git.

Setup overview

1. Create a new directory.
2. Install the Cognite Toolkit.
3. Use cdf modules init to initialize the configuration files.
4. Use cdf repo init to convert the directory to a Git repository and get CI/CD pipelines.
5. Use cdf auth init to set up and verify authentication for the Cognite Toolkit.

Step 1: Create a working root directory

The working directory serves as the root folder, and can later be converted to a Git repository. To create the working directory, open up a terminal and run this command specifying the directory name:

mkdir <your-working-root-directory>

Step 2: Install the Cognite Toolkit from PyPi

The Cognite Toolkit is available as a Python package and can be installed with pip or Poetry, ideally by using a virtual environment. Open up a terminal and install the Cognite Toolkit:

pip

cd <your-working-root-directory>

pyenv shell 3.11 # optional, but recommended
pip install cognite-toolkit
cdf --version

poetry

cd <your-working-root-directory>

poetry init . # please follow the interactive setup.
poetry add cognite-toolkit
poetry run cdf --version

This will add a pyproject.toml file to that can be checked in to Git to track dependencies.

If you're using a virtual Python environment manager, and you get a "command not found" message in the terminal window, make sure you have activated the virtual environment, for example by using source .venv/bin/activate or Poetry.

Step 3: Initialize the configuration files and modules

The Cognite Toolkit manages one or more Cognite Data Fusion projects that belongs to an organization. The organization name is the same as you will find in the browser address field after signing in to the Cognite Data Fusion web interface: https://<organization>.fusion.cognite.com.

To initialize the configuration files and modules, run this command:

cdf modules init <organization_dir>

This starts the interactive setup process to create the organization_dir directory and populate it with a set of configuration files and template modules to configure for your CDF projects.

For each organization the most common setup is to have one project per environment. In this example, the organization is called "acme" and the environments are "dev" and "prod":

Step 4: Convert the working directory to a Git repository

To prepare for version control and collaboration, convert the working directory into a Git repository:

git init .
cdf repo init

The directory can now be version-controlled and is ready for further configuration. The cdf repo init command adds a .gitignore file and a README.md file to the repository. Use the README file to document the purpose of the repository and the configuration files.

Step 5: Set up authentication and authorization

See the authentication and authorization article.