config.[env].yaml configuration file

Expected location: config.[env].yaml adjacent to the modules/ directory.

The config.[env].yaml file configures the modules for a specific environment. It's in YAML format in the same directory as the modules/ directory. You'll likely have at least one config.[env].yaml for each CDF project in your organization.

Example config.[env].yaml file

environment:
name: dev
project: demo
type: dev
selected: 
  - modules/

variables:
  modules:
    cdf_ingestion:
      workflow: ingestion
      groupSourceId: b89b117a-9f2e-4588-b7dc-bee39f2258d8
      pandidContextualizationFunction: contextualization_p_and_id_annotater
      contextualization_connection_writer: contextualization_connection_writer
    instanceSpaces:
      - springfield_instances
      - cdf_cdm_units

The environment section

The environment section specifies the configuration of a CDF project.

• name (string): a descriptive name of the environment. The Cognite Toolkit doesn't use the string.

• project (string): the name of the CDF project.

• validation-type (string): the type of environment, for example, dev, test, staging, qa, or prod. The toolkit strictly validates all environments except dev. For instance, if the environment variable CDF_PROJECT doesn't match the project value, the toolkit throws an error in all environments except dev.

• selected (list): a list of modules to include in the configuration. You can override the list using the --modules/-m flag in the build command.

  An entry in the list can be either a module name, for example, cdf_ingestion, or a path. If it's a path, it must be relative to the organization directory.

  If you specify a path, all modules in subdirectories are included. Using modules/ as an entry will include all modules in the modules/ directory and all subdirectories.

The variables section

The variables section contains variables used by the cdf build command.

You can use variables to define values used in multiple places in the configuration. For example, when creating a data model, it's helpful to specify the space as a variable since it's also referred to in containers, views, nodes, and data models. If the space name changes, you only need to update it in one place.

You can also use variables to define values that differ depending on the environment. For example, when you set up authentication, you can have a variable for sourceId that differs in dev and prod CDF projects.

The toolkit uses the variables when you create module templates with either cdf modules init or cdf modules add to show which values must be filled in by the user.

Specifying variables

Variables are organized as key-value pairs. This example has a variable named space with the value my_space;

variables:
  modules:
    space: my_space

If you have a data model that uses the space variable, you can reference it like this:

space: {{ space }}}
externalId: MyDataModel
version: v1
views:
 ...

When you run cdf build, the toolkit replaces {{ space }} with my_space in the data model configuration, and puts the results in the build/ directory.

space: my_space
externalId: MyDataModel
version: v1
views: ...

Variables structure

You can organize the variables section to match the module directory structure, for example, if you have variables that are only used by a single module, and other variables that are used by multiple modules.

For example, you can have this module directory structure:

./modules/
   ├── cdf_ingestion/
   │     ├── auth/
   │     ├── data_models/
   │     └── workflow/
   └── industrial_tools/
         └── cdf_search/
             ├── auth/
             ├── data_models/
             └── locations/

And this variables section in the config.[env].yaml file:

variables:
  modules:
    space: my_space
    cdf_ingestion:
      sourceId: 1234
    industrial_tools:
      cdf_search:
        sourceId: 5678

The example has three variables: space, sourceId for cdf_ingestion, plus sourceId for cdf_search.

The space variable is used by all modules, cdf_ingestion and cdf_search in this example.

The sourceId for cdf_ingestion is used only by the cdf_ingestion module, and the sourceId for cdf_search is used only by the cdf_search module.