Environment variables

The Cognite Toolkit supports injecting environment variables into the configuration files. Use environment variable injection to inject secrets or other sensitive information that you don't want to store in the configuration files.

The Cognite Toolkit injects environment variables in the cdf deploy/clean/pull commands. You don't need to declare environment variables. The Cognite Toolkit will inject all the variables into the OS environment. The syntax for injecting environment variables is ${MY_ENV_VAR} and you can place it anywhere in the configuration file.

For example, if you have the following configuration file for a workflow trigger:

my_trigger.WorkflowTrigger.yaml
externalId: my_trigger
triggerRule:
  triggerType: schedule
  cronExpression: '0 0 * * *'
metadata:
  origin: cognite-toolkit
workflowExternalId: wf_my_workflow
workflowVersion: '1'
authentication:
  clientId: ${IDP_WF_TRIGGER_CLIENT_ID}
  clientSecret: ${IDP_WF_TRIGGER_SECRET}

Then, in your environment, you'll have the following environment variables:

IDP_WF_TRIGGER_CLIENT_ID=my_client_id
IDP_WF_TRIGGER_SECRET=my_secret

When you run the cdf deploy command, the Cognite Toolkit injects the environment variables into the configuration file.

my_trigger.WorkflowTrigger.yaml
externalId: my_trigger
triggerRule:
  triggerType: schedule
  cronExpression: '0 0 * * *'
metadata:
    origin: cognite-toolkit
workflowExternalId: wf_my_workflow
workflowVersion: '1'
authentication:
  clientId: my_client_id
  clientSecret: my_secret

attention

The Cognite Toolkit doesn't inject environment variables for ExtractionPipelineConfig in the config parameter. This is because the extraction pipeline configurations and the Cognite Toolkit use the same syntax for injecting environment variables.

Environment variables with build variables

The environment variables are independent of the build variables specified in the config.[env].yaml file and used in the cdf build command. You can combine the environment and build variables together.

For example, you can define a function schedule in modules/my_module/functions/my_function.Function.yaml:

my_function.Function.yaml
name: daily-8am
cronExpression: '0 8 * * *'
functionExternalId: my_function
authentication:
  clientId: {{ functionClientId }}
  clientSecret: {{ functionClientSecret }}

Then, in your config.dev.yaml file, you'll have the following build variables:

config.dev.yaml
variables:
  modules:
    functionClientId: ${IDP_FUNCTION_CLIENT_ID}
    functionClientSecret: ${IDP_FUNCTION_CLIENT_SECRET}

When you run cdf build command, the Cognite Toolkit replaces the {{ functionClientId }} and {{ functionClientSecret }} with the environment variable placeholders. Hence, after running cdf build, your function schedule configuration in build/functions/1.my_function.Function.yaml will look like:

my_function.Function.yaml
name: daily-8am
cronExpression: '0 8 * * *'
functionExternalId: my_function
authentication:
  clientId: ${IDP_FUNCTION_CLIENT_ID}
  clientSecret: ${IDP_FUNCTION_CLIENT_SECRET}

In cdf deploy command, specify the environment variables, IDP_FUNCTION_CLIENT_ID and IDP_FUNCTION_CLIENT_SECRET. The Cognite Toolkit will inject these environment variables into the configuration file.

my_function.Function.yaml
name: daily-8am
cronExpression: '0 8 * * *'
functionExternalId: my_function
authentication:
  clientId: my_client_id
  clientSecret: my_secret

Environment variables with build variables​

Environment variables with build variables