Skip to main content
An external data source is a reusable, named connection that lets Transformations read from storage outside Cognite Data Fusion (CDF). For example, you can read Microsoft Fabric OneLake or Snowflake without staging a full copy in CDF RAW. Configure the connection once, then reference the source in transformation SQL by externalId. This API is enabled per project. If requests return that the endpoint isn’t enabled, contact Cognite Support.

Concepts

  • Source type — The external system the source connects to. Supported values are one_lake for Microsoft Fabric OneLake and snowflake for Snowflake.
  • OneLake credentials — Microsoft Entra ID application credentials: clientId, tenantId, and clientSecret. The client secret is write-only. CDF stores it encrypted and never returns it.
  • OneLake location — Where in OneLake the source points: workspaceId and containerId, both as GUIDs.
  • Snowflake credentials — accountIdentifier, userName, and roleName. CDF generates an RSA key pair, stores the private key encrypted, and returns only the public key. Register that public key on the Snowflake service user. Set expiryTime in epoch milliseconds, in the future, and at most one year ahead.
  • Snowflake location — warehouseName.
  • Data set scoping — Optional dataSetId that scopes access to the data source through CDF data set permissions.

Usability checks

Before you use an external data source in a transformation, call the usability endpoint (POST /transformations/externaldata/usability) to verify that CDF can reach the storage location with the stored credentials. The response reports whether the connection and permissions succeed, so you can fix configuration issues before a transformation job fails.

Retrieve by ID

Retrieve returns items in the order of the requested external IDs. Each externalId in the request must be unique. If an ID is missing or you cannot access that source, the request fails with HTTP 400 and lists the missing IDs. Set ignoreUnknownIds to true to omit those IDs instead.

Rotate keys

Rotate replaces the RSA key pair and sets the new expiry. The call is atomic: if one item fails, no key is rotated. Only Snowflake sources can be rotated. Another source type rejects the whole request with HTTP 400.

Key capabilities

  • List external data sources configured for the project
  • Create external data sources with a unique externalId
  • Retrieve external data sources by external ID
  • Rotate keys on Snowflake sources
  • Delete external data sources by id or externalId
  • Verify usability of a data source before using it in a transformation
If the externalId already exists in the project, POST /transformations/externaldata returns 409 and creates no items. To change a source, delete it and create a replacement. There is no update operation.
The stable (20230101) API reference does not include external data sources. Use the beta or alpha API reference for concepts and endpoint documentation. Send Cdf-Version: beta on requests. For how preview APIs relate to stable, see Beta API reference.

Rate and concurrency limits

External data source operations share the general CDF request budgets. For limit types and throttling behavior, see API rate limits.
Last modified on November 17, 2026