Manage data models in CI/CD

Beta

The features described in this section are currently in beta testing and are subject to change.

This article describes how to use the Cognite Data Fusion (CDF) CLI to manage a data model in a CI/CD pipeline with GitHub actions.

Prerequisites

Before you start, make sure you have completed these steps:

1. Install the CDF CLI on your machine.
2. Created your data model.

Step 1: Create a GitHub Action

1. Create a GitHub Action to run every time you merge to the main branch.

   .github/workflows/publish-dm-on-main.yml

   name: Publish datamodel on merge to main
   on:
     push:
       branches:
         - main
   jobs:
     publish-dm:
       runs-on: ubuntu-latest
       steps:
         - name: Check out code
           uses: actions/checkout@v3
   
         - name: Setup Node
           uses: actions/setup-node@v3
           # We will add the next steps here

Step 2: Install the CDF CLI

1. Add commands to install the CDF CLI in the GitHub Action.

   - name: Install
     run: |
       npm install --global @cognite/cdf-cli

Step 3: Sign in with the CDF CLI

1. In GitHub, set up the secrets.

2. Add commands to sign in to CDF with the CDF CLI.

   - name: Signin
     run: |
       cdf signin ${{ secrets.CLI_PROJECT }} --tenant=${{ secrets.CLI_TENANT }} --cluster=${{ secrets.CLI_CLUSTER }} --client_id=${{ secrets.CLI_CLIENT_ID }} --client_secret=${{ secrets.CLI_CLIENT_SECRET }}

   Where:

   • CLI_PROJECT - The CDF project (for example, demo).
   • CLI_TENANT - The Microsoft Entra ID (formerly Azure Active Directory) tenant (for example, cognitecli.onmicrosoft.com).
   • CLI_CLUSTER - The CDF cluster (default: api).
   • CLI_CLIENT_ID - A client ID for an application registered in your Microsoft Entra ID tenant.
   • CLI_CLIENT_SECRET - A secret for an application registered in your Microsoft Entra ID tenant.

note

We currently support using Microsoft Entra ID to sign in to the CDF CLI.

Step 4: Publish the data model

1. Add commands to publish the data model.

   In the example below the data model lives in datamodel.graphql within the root of the repo and you publish changes to version 1 of a data model called moviedm in the myspace space.

   - name: Publish changes
     run: |
       cdf data-models publish --file=$file --external-id=$external_id --space=$space --version=${version:-${GITHUB_SHA:0:6}}
     env:
       file: './datamodel.graphql'
       external_id: 'moviedm'
       space: 'myspace'
       version: '1'

Using the git hash for versioning

Since we support all types of text for the version, you can also use the git hash for the version. In the above example, if you just remove version from env, then it will default to using the SHA for versioning instead.

By default, in github action, it's ${GITHUB_SHA:0:6}. The :0:6 just takes the first 6 character of the SHA.

Sample GitHub Action file

.github/workflows/publish-dm-on-main.yml

name: Publish DM on merge to main
on:
  push:
    branches:
      - main
jobs:
  publish-dm:
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3

      - name: Install
        run: npm install --global @cognite/cdf-cli

      - name: Sign in
        run: |
          cdf signin ${{ secrets.CLI_PROJECT }} --tenant=${{ secrets.CLI_TENANT }} --cluster=${{ secrets.CLI_CLUSTER }} --client_id=${{ secrets.CLI_CLIENT_ID }} --client_secret=${{ secrets.CLI_CLIENT_SECRET }}

      - name: Publish changes
        run: |
          cdf data-models publish --file=$file --external-id=$external_id --space=$space --version=${version:-${GITHUB_SHA:0:6}}
        env:
          file: './datamodel.graphql'
          external_id: 'moviedm'
          space: 'myspace'
          version: '1' # if you want to use the git SHA, then you can remove this