Designing scalable data models

This guide outlines a strategic approach to designing scalable and efficient data models in Cognite Data Fusion (CDF). It covers layered architecture (source, enterprise, solution), containers, views, design principles, and deployment practices for maintainability, scalability, and security.

​

Why it matters

A well-designed data model reduces integration complexity, improves query performance, and makes it easier to scale as your organization grows. Poor design leads to slow queries, maintenance overhead, and difficulty adding new data sources or applications. Following the principles in this guide helps you avoid common pitfalls and build models that support both operational and analytical use cases.

​

Layered architecture

A layered data model architecture separates responsibilities and optimizes for different use cases. Clear ownership at each layer improves maintainability: Data producers focus on quality and freshness, data modelers on integration and business logic, and data consumers on application-specific views. This structure also scales better as your organization grows, because changes in one layer do not cascade unpredictably to others. The three layers map loosely to the medallion architecture (bronze, silver, gold). See medallion architecture for an overview. As a general rule, map data into solution views directly from enterprise data model containers rather than transforming or duplicating the data. In specific cases, a solution model can include its own containers (populated by discrete workflows) to store data unique to a particular use case or application. Write-back principle: For write-back use cases, data must be written to a separate source data model (owned by a data producer), never directly into the enterprise layer.

​

Core design principles

Beyond layered architecture, a few design principles keep your data models efficient and scalable. These apply whether you are building a source, enterprise, or solution model.

​

Avoid duplication via shared IDs

A single entity should be represented once in the knowledge graph. Data across views and layers is flexibly linked by using the same instance ID (space + external ID). Explicit direct relations are typically only needed to enable navigation in the UI/GraphQL or when linking instances that have different IDs.

​

Extend core data models (CDM)

Custom views must extend (implement) the relevant base core data model (CDM) types rather than using original CDM views directly. Relating CDM views to custom views that extend CDM is complicated and risks breaking the schema and losing relations. Include all relevant CDM types for the best GraphQL and CDF service compatibility, but omit those not utilized (for example, exclude Cognite3DModel if there is no 3D model). Extend CogniteAsset only once, with a single Asset type, and provide links to domain-specific dimensions in a direct relation. A top-level central type makes the model easy to navigate and query, allows implementation of specific hierarchies within related sub-types, and provides flexibility to link documents and time series to all levels. Avoid single-purpose wide and sparse views or multiple extensions of the CogniteAsset type.

​

Distribute properties contextually

Split properties for the same instance across multiple, context-specific containers and views. This ensures compliance with the 300-property view limit and improves AI accuracy by restricting the view to only context-relevant properties. For example, a property like pipe coating would be excluded from a view focused on pump equipment.

​

Model relationships

Choose the right relationship type for your use case. Direct relations (One:One, One:Many, Many:Many) are the default: they are simpler, faster, and less resource-intensive. Always include a reverse direct relation. Use edges only when you need relationship properties, when you must exceed the 2,000 linked instance limit, or when you must traverse the graph in queries and attach extra properties to the relationship. Edges add complexity and cost.

​

Query and performance optimization

Indexing and constraints are essential for efficiency and to prevent timeouts. Without them, queries on large graphs can become unusably slow. Add B-tree indexes to scalar properties and direct relations used for filtering or reverse traversal. Use inverted indexes for list-type properties. Note the limit of 10 indexes per container. See Performance considerations for indexing guidance. Use human-readable names and descriptions for all properties to aid users and AI agents (Atlas AI). Denormalize properties that are frequently queried and give additional context, since this helps AI agents, applications, and human users alike. See Optimizing data models for AI for more guidance.

​

Security and governance

Separating schema from data instances enables granular access control. Use distinct schema spaces (for model definitions) and instance spaces (for data), and rely on tooling for controlled deployment. The permission model distinguishes schema from data: datamodels:read gives schema access, while datamodelinstances:read is required to read any data. Time series and files are special cases. Conceptually, they are meta/container types: the data model instance (the node) points to the actual bulk storage for the instance’s data (the datapoints or file bytes). To control access, you need two distinct types of permissions:

1. Schema access: You need datamodels:read on the cdf_cdm space. This is required for your application to understand the structure and definition of what a CogniteTimeSeries or CogniteFile is.
2. Data access: You need datamodelinstances:read on the instance space (for example, my-asset-space). This grants access to read the specific instances and their associated time series datapoints or file bytes. Using distinct instance spaces is the essential strategy for segregating data access.

Separate instances by creating a space for each location (for example, sp_location_a_instances) to control data access using CDF groups.

​

Source and enterprise layer instance strategy

Define a strategy for instances between the source and enterprise layers. In the solution layer, you generally map data within the enterprise layer. Separation prevents data deletion risk and enables data isolation (for example, for third-party suppliers who may be competitors) but increases the instance count. Combining reduces instances and simplifies mapping, but risks unintended data loss. Extra care is needed when making changes in the source layer, since deleting an instance in a source model would delete the same instance in the enterprise model.

​

Deployment

Use the Cognite Toolkit as the primary tool for managing and deploying data models. Store models in YAML format in a source code management platform (Git) to enable versioning, schema validation, and deployment across environments (Dev, Staging, Prod) using a CI/CD pipeline. This workflow gives you change control, traceability, and standardized deployments (for example, Gitflow). Consider NEAT when domain subject matter experts (SMEs) need to be involved in both model design and implementation. NEAT simplifies extending CDM types, creating enterprise models that can be subdivided into solution models, and configuring performance optimizations such as indexes and constraints. It may be less useful when most work is done by experienced data modelers who may not need these simplifications.

​

Naming and versioning

Consistent naming and versioning make long-term maintenance easier and simplify integration across teams. Follow the resource naming conventions. Key recommendations:

• Use PascalCase for external IDs of containers and views (for example, CentrifugalPump).
• Use camelCase for property identifiers (for example, ratedCurrent).
• Avoid company names in external IDs.

Use a numerical versioning scheme (v1, v1.0.0) and bump the view version with every schema change (recommended). Use Cognite Toolkit variables to configure names (for example, space names) and versions, avoiding the need to manually update all YAML configurations when changes occur.

​

Validation checklist

Before deploying or merging changes, run through this checklist to confirm your model follows the best practices in this guide. The following table lists verification items by category: layered architecture, core design, query and performance, governance, deployment, and naming.

​

Tools and services

The following services and tools support data modeling in CDF. Choose the right tool based on your role and workflow.

​

Further reading

• Data modeling concepts – Core concepts: property graph, spaces, instances, containers, views, and data models.
• Data modeling guides – Examples and best practices for extending CDM, performance, and CI/CD.
• Data modeling principles – Organizational principles and best practices for data modeling with NEAT.
• Resource naming conventions – Complete reference for naming CDF resources.