Setting up the Cognite Toolkit
Follow the steps in this article to install the Cognite Toolkit and the basic file structure on your computer and prepare the configuration for integration with version control repository.
We highly recommend that you use Git and a service like GitHub or Azure DevOps to manage the project configurations "as code". For governance of production projects, we also recommend using a CI/CD pipeline to deploy changes to Cognite Data Fusion.
Prerequisites
- A recent version of Python, at minimum v3.10.
- Recommended: a virtual environment manager like Poetry or pyenv for Python versions and dependencies.
- Recommended: Basic knowledge of Git.
- Create a new directory.
- Install the Cognite Toolkit.
- Use
cdf modules init
to initialize the configuration files. - Use
cdf repo init
to convert the directory to a Git repository and get CI/CD pipelines. - Use
cdf auth init
to set up and verify authentication for the Cognite Toolkit.
Step 1: Create a working root directory
The working directory serves as the root folder, and can later be converted to a Git repository. To create the working directory, open up a terminal and run this command specifying the directory name:
mkdir <your-working-root-directory>
Step 2: Install the Cognite Toolkit from PyPi
The Cognite Toolkit is available as a Python package and can be installed with pip or Poetry, ideally by using a virtual environment. Open up a terminal and install the Cognite Toolkit:
- pip
- poetry
cd <your-working-root-directory>
pyenv shell 3.11 # optional, but recommended
pip install cognite-toolkit
cdf --version
cd <your-working-root-directory>
poetry init . # please follow the interactive setup.
poetry add cognite-toolkit
poetry run cdf --version
This will add a pyproject.toml file to that can be checked in to Git to track dependencies.
If you're using a virtual Python environment manager, and you get a "command not found"
message in the terminal window, make sure you have activated the virtual environment, for example by using source .venv/bin/activate
or Poetry.
If the installation fails, see the troubleshooting section.
Step 3: Initialize the configuration files and modules
The Cognite Toolkit manages one or more Cognite Data Fusion projects that belongs to an organization. The organization name is the same as you will find in the browser address field after signing in to the Cognite Data Fusion web interface: https://<organization>.fusion.cognite.com
.
To initialize the configuration files and modules, run this command:
cdf modules init <organization_dir>
This starts the interactive setup process to create the organization_dir directory and populate it with a set of configuration files and template modules to configure for your CDF projects.
For each organization the most common setup is to have one project per environment. In this example, the organization is called "acme" and the environments are "dev" and "prod":
Step 4: Convert the working directory to a Git repository
To prepare for version control and collaboration, convert the working directory into a Git repository:
git init .
cdf repo init
The directory can now be version-controlled and is ready for further configuration. The cdf repo init
command adds a .gitignore
file and a README.md
file to the repository. Use the README file to document the purpose of the repository and the configuration files.
Step 5: Set up authentication and authorization
See the authentication and authorization article.
Troubleshooting
I get an error message when installing the Cognite Toolkit
If you are a Windows user, you might get the following error: ERROR: Could not install package due to an OSError: [Errno 2] No such file or directory: ...
. It means that you don't have Windows Long Path support enabled.
The Cognite Toolkit comes installed with built-in modules. These modules are part of a directory structure that can exceed the Windows limit of 260 characters for a file path, depending on the location where you install the toolkit.
To enable Long Path support, see the maximum path length limitation article.