> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognite.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Cognite Data Fusion (CDF) is an industrial DataOps platform. Prefer English docs unless the user asks for another locale.
> Follow /_llms/ index links recursively until you reach page URLs ending in .md. Fetch those Markdown twins instead of HTML.
> For REST APIs, default to calendar version 20230101 (stable). Use 20230101-beta or 20230101-alpha only if the user is on a preview API. Cite paths under /api-reference/concepts/<version>/ and the Cdf-Version header. Prefer the Python SDK (cognite-sdk) unless the user is in JavaScript/TypeScript.
> Start here by task: platform overview /cdf/index.md; data modeling /cdf/dm/index.md; data integration /cdf/integration/index.md; access /cdf/access/index.md; CDF Toolkit /cdf/deploy/cdf_toolkit/index.md; REST quickstart /dev/quickstart.md; Python SDK https://cognite-sdk-python.readthedocs-hosted.com/en/stable/; Atlas AI /cdf/atlas_ai/concepts/index.md; Flows apps /cdf/flows/index.md; MCP and IDE setup /dev/guides/ide_ai_integration.md.
> For implementation work, also load /skill.md.

# Working with modules

> List modules, pull Cognite Data Fusion (CDF) edits back into YAML, add a deployment pack, and upgrade Cognite Toolkit project files.

`cdf modules` lists the modules in your project, pulls CDF changes back into those modules, adds deployment packs, and upgrades project files when you install a new Cognite Toolkit version.

## List modules

Use this command to see which modules are in the `modules/` directory and whether the Cognite Toolkit found any issues. Data engineers and Cognite Toolkit administrators both run it.

`cdf modules list` prints one row per module:

```shell title="Module status" theme={"languages":{"custom":["/_languages/kuiper.json"]}}
Module details
 Module     Status    Resources   Insights                   Syntax warnings
─────────────────────────────────────────────────────────────────────────────
 access     SUCCESS           1   ✓                                        0
 my_module  SUCCESS          23   ✓                                        0
```

**SUCCESS** and a check mark under Insights mean the module is ready to build and deploy.

## Pull modules

Use `cdf modules pull` to copy resources from your CDF project back into `modules/`. The command keeps the module structure, variables, and comments. It only updates resources that are already in a module. It is the reverse of `cdf build` and `cdf deploy`. Pull after you edit a resource in the CDF UI and want that change in the module.

Before you pull, read [Building configurations](https://docs.cognite.com/cdf/deploy/cdf_toolkit/getting_started/build.md) and [Deploying configurations](https://docs.cognite.com/cdf/deploy/cdf_toolkit/getting_started/deploy.md). Keep a `.env` file for your CDF project.

**1. Create a bare-bones transformation**

Build and deploy a transformation to CDF. Create this YAML file:

```yaml title="Bare-bones transformation" theme={"languages":{"custom":["/_languages/kuiper.json"]}}
externalId: {{transformationPrefix}}myTransformation
name: myTransformation
ignoreNullFields: true # Keep null fields from the source data.
```

and place it in `modules/my_module/transformations/my.Transformation.yaml`.

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
modules/
  my_module/
    transformations/
      my.Transformation.yaml
```

Build and deploy the transformation:

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
cdf build --config-yaml config.dev.yaml
```

and

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
cdf deploy
```

**2. Edit the transformation in CDF**

Open the transformation in CDF and add this `query`:

```sql theme={"languages":{"custom":["/_languages/kuiper.json"]}}
select * from cdf_nodes('cdf_cdm', 'CogniteDescribable', 'v1')
```

**3. Pull the transformation into your local module**

Pull the transformation from CDF into the local module:

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
cdf modules pull --config-yaml config.dev.yaml my_module
```

The command writes the CDF change into the local module and keeps the module structure, variables, and comments. The summary reports one changed transformation.

**4. Inspect result**

Open the transformation YAML:

```yaml title="Pulled transformation" wrap theme={"languages":{"custom":["/_languages/kuiper.json"]}}
externalId: {{transformationPrefix}}myTransformation
name: myTransformation
ignoreNullFields: true # Keep null fields from the source data.
query: select * from cdf_nodes('cdf_cdm', 'CogniteDescribable', 'v1')
```

The `query` you added in CDF is now in the local module. The variable, the comment, and the rest of the file are unchanged.

If you use `git` or another version control system, you should now commit the changes to your local module.

## Add deployment packs

Data engineers rarely write every resource from scratch. Start from a deployment pack: a templatized module you adapt for your project. See [Deployment packs](https://docs.cognite.com/cdf/deploy/cdf_toolkit/references/packages.md).

**1. Set the deployment pack source**

Put `cdf.toml` in the directory where you run `cdf`. The file includes the source of the deployment packs.

```toml title="Example cdf.toml file" wrap theme={"languages":{"custom":["/_languages/kuiper.json"]}}
...

[library.cognite]
url = "https://github.com/cognitedata/library/releases/download/latest/packages.zip"
```

This is the default source of deployment packs. Cognite develops and maintains that library.

**2. Add a deployment pack**

To add a deployment pack, run:

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
cdf modules add
```

If you do not have a `modules/` directory, run `cdf modules init` instead. `cdf init` runs that command when you select **Modules**.

The wizard asks you to select and install a deployment pack:

```shell title="Select a deployment pack" theme={"languages":{"custom":["/_languages/kuiper.json"]}}
$ cdf modules add
Adding library cognite from https://github.com/cognitedata/library/releases/download/latest/packages.zip
Download ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Unzipping ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

? Which package would you like to use? Common: Core platform setup: common resources, ingestion pipeline, and search. (3)
   Which modules in common would you like to add? [Ingestion Workflow]

You have selected the following:

     modules
     └── common
         └── cdf_ingestion


? Would you like to make changes to the selection? No
  Creating common
    Creating module cdf_ingestion
  Updating config.dev.yaml
cdf.toml file already exists. Skipping creation.
```

**3. Inspect the results**

After the command has completed, you can inspect the results. Your `modules/` directory should now contain
the new module you added:

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
modules/
  common/
    cdf_ingestion/
      auth/
      data_sets/
      workflows/
      module.toml
      README.md
```

Each module has a `module.toml` file that describes the source of the module. Keep this file in the module
directory. The `README.md` file contains documentation from the author of the deployment pack.

Then inspect `config.dev.yaml`. If the deployment pack defines variables, `config.dev.yaml` is updated:

```yaml title="Example config.dev.yaml file" theme={"languages":{"custom":["/_languages/kuiper.json"]}}
environment:
  ...

variables:
  ... Previous variables
  modules:
    common:
      cdf_ingestion:
        workflow: ingestion
        dataset: ingestion
        ...
```

The variables `modules.common.cdf_ingestion.workflow` and `modules.common.cdf_ingestion.dataset` are now
in `config.dev.yaml`.

**4. Next steps**

The first step after adding a deployment pack is to update the variables to match your use case. Then
[build](https://docs.cognite.com/cdf/deploy/cdf_toolkit/getting_started/build.md) and [deploy](https://docs.cognite.com/cdf/deploy/cdf_toolkit/getting_started/deploy.md) the resources in the module.

## Upgrade modules

Cognite Toolkit administrators run `cdf modules upgrade` after installing a new CLI version. The command updates
`cdf.toml` and CI/CD pipeline files so they match that version.

```shell theme={"languages":{"custom":["/_languages/kuiper.json"]}}
cdf modules upgrade --verbose
```

`--verbose` prints each change. A version change updates the Cognite Toolkit version in `cdf.toml`:

```toml title="cdf.toml" theme={"languages":{"custom":["/_languages/kuiper.json"]}}
[modules]
version = "0.9.24"
```

The same run updates the image tag in the workflow files from `cognite/toolkit:0.9.23` to `cognite/toolkit:0.9.24`. Review the diff before you commit it.

> [!NOTE]
>
> When a resource format has a breaking change, `cdf modules upgrade` tries to rewrite the YAML files to the new format. Review that diff before you commit it.