> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognite.com/llms.txt
> Use this file to discover all available pages before exploring further.

# About transformations

> Learn about data transformation in Cognite Data Fusion (CDF) and how to use built-in transformation tools to reshape, enrich, contextualize, and analyze data quality in your data integration workflow.

## Transformation architecture and purpose

A transformation specifies how source data is mapped and written to a target structure in CDF.

Transformations can enrich data with other sources or calculations, contextualize it by matching related objects, and check data quality before it reaches downstream users. A common pattern is to read staged data from CDF RAW and write it to a structured target such as a data model. This helps you deliver consistent, queryable data to apps and workflows, for example when you standardize equipment metadata before it is used in dashboards.

```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
flowchart LR
  subgraph Sources
    A[Source systems]
  end

  subgraph Staging
    B[RAW staging]
  end

  subgraph Processing
    C[[Transformations]]
    C1[(Spark SQL)]
    C --> C1
  end

  subgraph Targets
    D[Data models]
    E[Sequences]
    F[Files]
  end

  A --> B --> C
  C --> D
  C --> E
  C --> F

  classDef stage fill:#EEF6FF,stroke:#3B82F6,stroke-width:1px;
  classDef process fill:#F0FDF4,stroke:#16A34A,stroke-width:1px;
  classDef target fill:#FFF7ED,stroke:#EA580C,stroke-width:1px;
  class B stage;
  class C,C1 process;
  class D,E,F target;
```

Transformations are developer-centric tools for data engineers and developers who define schemas, write SQL, and manage pipelines. **CDF Transformations run on a managed Spark SQL engine.** You express logic in SQL (or map fields in the UI), and CDF handles scheduling, scaling, and access to CDF data sources.

* Start with SQL and use transformations when your logic is declarative and best expressed as set-based operations across tables.

* Use [CDF Functions](/cdf/functions/index) when you need Python logic, external libraries, or custom API calls.

* Use the [Cognite Toolkit](/cdf/deploy/cdf_toolkit) when you want to manage transformations as code. The Toolkit lets you define transformations, schedules, and notifications in YAML (with optional SQL files) and deploy them through CI/CD.

* You can also run transformations using the [Cognite API](/api-reference/concepts/20230101/transformations), and the [Cognite Python SDK](https://cognite-sdk-python.readthedocs-hosted.com/en/latest/transformations.html).

<Warning>
  Avoid transformations for high-volume or low-latency writes such as high-frequency datapoints. For these scenarios, prioritize direct ingestion pipelines or [Files](/api-reference/concepts/20230101/files) to keep latency and throughput within system limits.
</Warning>

## Target selection and operational constraints

Decide where data should land early, and avoid defaulting to the CDF staging area ([RAW](/dev/concepts/resource_types/raw)) as the final destination. Use RAW for staging, then write to a target that matches how the data will be used.

<Tip>
  Treat RAW as a staging layer and align the target with how the data will be used.
</Tip>

<CardGroup cols={2}>
  <Card title="Create a transformation" icon="gear" href="/cdf/integration/guides/transformation/transformations">
    Define and run your first transformation in CDF.
  </Card>

  <Card title="SQL syntax and functions" icon="code" href="/cdf/integration/guides/transformation/write_sql_queries">
    Reference for SQL syntax and custom functions.
  </Card>
</CardGroup>