Modeling an asset hierarchy
This article describes the asset hierarchy template in the Cognite Data Fusion (CDF) templates repository. An asset hierarchy describes connections between larger assets and the smaller components inside them. The tools and information in the in the tools are provided as-is, without any support or warranties,
We recommend that you read this article to become familiar with the asset hierarchy template before you try it in your Cognite Data Fusion (CDF) project.
The tools and templates in the Cognite Data Fusion (CDF) templates repository are community-supported with contributions from Cognite engineers.
The template first creates a generic asset data model and then a use-case-specific data model and demonstrates how to:
- Store data and data models in containers.
- Populate data models with transformation jobs.
- Use the existing system containers and views.
- Consume the data models with GraphQL and SDKs.
The template models an asset hierarchy with two levels, lift stations and pumps, where the pumps are children of a lift station:
Each of the pumps has properties in a metadata key-value store:
The data models
This section describes the views, the solution data model for the template. The containers, the physical storage, are covered in the next section.
Generic asset model
The template starts with a generic asset model. The data model has one view, Asset:
The data model is generic, and you can use it for both lift stations and pumps. However, you can't define properties specifically for pumps since the properties would also apply to lift stations. Therefore, the model is useful for creating an overview of all assets, but you need additional data models tailored to specific use cases.
Use-case-specific data model
The use-case-specific data model has two views, LiftStation and Pump. There's a one-to-many edge from LiftStation to Pump and a direct relation from Pump to LiftStation.
The data model fits the use case, and you can query it in a language close to the domain, for example, "get all pumps in lift station 1."
Note that while the data model is well suited for building a solution in the domain of pump lift stations, you likely can't use it for other use cases.
Implementing the physical schema backing your data model
When you create data models, you need to consider containers and how to store data. Containers handle the physical storage of data models and are similar to SQL tables.
While views are reasonably easy to create, modify, and delete, containers are more permanent, and any changes often require costly migration of data. Carefully consider which properties to include in each container, keeping the following questions in mind:
-
How can you avoid duplication of data? Duplication can lead to inconsistencies and increased storage costs.
-
What queries do you expect? Consider which indexes you need in the containers. Indexes speed up queries, but increase the cost of writing.
-
What structure do you want to enforce on the data? Defining required properties or requiring a matching row to exist in another container lets you protect data quality by rejecting write operations of incomplete data.
When you design containers, Cognite recommends using system containers and views as much as possible. They're pre-defined and designed to fit common industrial use cases and are available in the cdf_core
space.
Implementing the pump lift station use case
Generic asset model
The generic asset model is similar to the Asset
system view in the cdf_core
space. The only difference is the added metadata
property.
We don't recommend using a generic metadata
property of type JSON. See how the Asset
system view can be extended in the example.
To store the metadata
property, the template uses a new container, AssetMetadata
, in the myModelSpace
space. One potential issue is that the container could be used to store any information in the metadata
property. To make sure that it's only used for assets, we've added a constraint to the Asset
container in the cdf_core
system space.
The template defines the container specification in a YAML file and writes it to CDF using the container API.
externalId: AssetMetadata
name: Asset
space: myModelSpace
usedFor: node
properties:
metadata:
autoIncrement: false
defaultValue: null
description: Custom, application specific metadata. String key -> String value.
name: metadata
nullable: true
type:
list: false
type: json
constraints:
requiredAsset:
constraintType: requires
require:
type: container
space: cdf_core
externalId: Asset
Next, the template creates an Asset
view for the generic asset model. Because the view is in the myModelSpace
space, it won't collide with the system Asset
view.
Instead of creating one property in the new view for each property in the Asset
, Describable
, and Sourceable
containers, the template uses the Asset
view from the cdf_core
space, which already has all the properties. Only the metadata
property needs to be added:
externalId: Asset
name: Asset
space: myModelSpace
version: '1'
implements:
- type: view
space: cdf_core
externalId: Asset
version: v1
properties:
metadata:
container:
externalId: Asset
space: myModelSpace
type: container
containerPropertyIdentifier: metadata
name: metadata