> ## Documentation Index > Fetch the complete documentation index at: https://docs.cognite.com/llms.txt > Use this file to discover all available pages before exploring further. # Containers, views, polymorphism, data models > Learn how to define schemas using containers, views, and data models to bring your own properties into the knowledge graph. > For the complete Cognite documentation index, see [/llms.txt](/llms.txt). For full page content, see [/llms-full.txt](/llms-full.txt). You can create your own schemas to enrich instances with custom properties. The schemas consist of three key elements: * **Containers**: define physical storage which contains properties. * **Views**: establish logical schemas which map properties. * **Data models** are collections of one or more views, used for graph data consumption and ingestion. All three elements are scoped to a space, just like instances: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph TB classDef SpaceClass fill:#0969DA,stroke:#24292f,stroke-width:2px,color:#fff classDef ViewClass fill:#8250df,stroke:#24292f,stroke-width:2px,color:#fff classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Space([Space]) Container[[Container]] Property([Property]) Index([Index]) Constraint([Constraint]) View[[View]] DataModel([Data model]) Space:::SpaceClass --has-container--> Container Container:::ContainerClass --has-index --> Index Container --has-constraint--> Constraint Space --has-view--> View View:::ViewClass --maps-property--> Property Container --has-property--> Property Space --has-data-model--> DataModel DataModel --contains-view--> View ``` ## Containers Containers are the physical storage for **properties**. They are defined within a space, and hold a set of properties that logically belong together. You must define **types** for your properties, and you can add optional **constraints** that the data must adhere to, and define **indexes** to optimize query performance. You can also define **indexes** for properties or groups of properties to optimize query performance. Containers store properties for instances (nodes and edges.) An instance can have properties in multiple containers: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph TB classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Equipment[["Equipment
manufacturer (string)"]] Pump[["Pump
maxPressure (float)"]] Node(("Node
externalId: (string)")) Equipment:::ContainerClass --has-data--- Node Pump:::ContainerClass --has-data--- Node ``` You can populate the containers for an instance, in this example below, for a node. This data: ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} externalId: 'xyz42' equipment: manufacturer: 'Acme Inc.' pump: maxPressure: 1.2 ``` translates to this: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph LR classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff subgraph types PumpTypeNode(("PumpTypeNode
externalId: pump")) end subgraph equipment Equipment[["Equipment
manufacturer: 'Acme Inc.'"]] Pump[["Pump
maxPressure 1.2"]] Node(("PumpNode
externalId: xyz42
type: [types, pump]")) Equipment:::ContainerClass --has-data--- Node Pump:::ContainerClass --has-data--- Node Node -.-> PumpTypeNode end ``` You can define containers in different space than the space holding the instances. This can be useful if you want to use the same schema for nodes in different spaces, which is often the case given the [access control model](/cdf/dm/dm_concepts/dm_access_control). ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph LR classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff subgraph types PumpTypeNode(("PumpTypeNode
externalId: pump")) end subgraph schema Equipment[["Equipment
manufacturer: (string)"]] Pump[["Pump
maxPressure (float)"]] end subgraph equipment1 Node(("PumpNode
externalId: xyz42
type: [types, pump]
manufacturer: 'Acme Inc.'
maxPressure 2.3")) Equipment:::ContainerClass --has-data--- Node Pump:::ContainerClass --has-data--- Node Node -.-> PumpTypeNode end subgraph equipment2 Node2(("PumpNode
externalId: abc123
type: [types, pump]
manufacturer: 'Other Inc.'
maxPressure 5.6")) Equipment:::ContainerClass --has-data--- Node2 Pump:::ContainerClass --has-data--- Node2 Node2 -.-> PumpTypeNode end ``` As you add data to these containers for more nodes, the physical storage of the containers will look similar to this: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph LR classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Node[[" Node
(space, externalId, type, ...)
(equipment1, xyz42, [types, pump], ...)
(equipment2, abc123, [types, pump], ...)
(types, pump, null, ...)
... "]] Equipment[[" Equipment
(space, externalId, manufacturer)
(equipment1, xyz42, Acme Inc.)
(equipment2, abc123, Acme Inc.)
"]] Pump[[" Pump
(space, externalId, maxPressure)
(equipment1, xyz42, 20.7)
(equipment2, abc123, 14.2)
"]] Equipment:::ContainerClass --> Node:::ContainerClass Pump:::ContainerClass --> Node Pump -.-> Equipment ``` Note that only `node.{space, externalId, type}` is included in the `Node` base container for brevity. This is similar to relational database schemas where `(space, externalId)` constitutes a foreign key to the core node table, and results in a [snowflake schema](https://en.wikipedia.org/wiki/Snowflake_schema). Importantly, this data lives on a different plane than the graph data discussed in the previous section. For example, nothing ensures that a node has data in `Pump` just because it has `node.type` set to `[types, pump]`. Validation of data content is left to the client to determine, but you can use [**views**](#views) to make it more ergonomic. ### Which types of instances can you use a container for? The `usedFor` field lets you define which types of instances the containers can be used for. Specify one of these values: * `node`: the container can only be used to populate properties on nodes. * `edge`: the container can only be used to populate properties on edges. * `all`: the container can be used to populate properties on both nodes and edges. * `record`: the container can only be used for [records](/cdf/dm/records/). You can only ingest records into containers with `usedFor` set to `record`. Containers designated for records support significantly more properties than standard containers. See [Limits and restrictions](/cdf/dm/dm_reference/dm_limits_and_restrictions). If you use `all`, ingesting to the container will be more expensive than using only `node` or `edge`. ### Properties When you define a container, you must specify the properties it will contain. Data modeling supports the following basic data types for properties: | Property type | Description | | ------------- | -------------------------------------- | | `text` | A string of characters. | | `int64` | A 64-bit integer. | | `float64` | A 64-bit floating point number. | | `float32` | A 32-bit floating point number. | | `boolean` | A boolean value. | | `timestamp` | A timestamp (with timezone). | | `date` | A date (without timezone). | | `json` | A JSON object. | | `direct` | A direct relation to another instance. | | `enum` | A fixed set of named values. | In addition to these property types, we support native reference types that point to resources in other Cognite APIs. This lets you reference data not suited for storage in a property graph. We support the following native resource reference types: | Native resource reference type | Description | | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | | `TimeSeries` | A reference to one specific time series. You can use GraphQL queries to expand data from the time series, including data points. | | `File` | A reference to a file stored in CDF and uploaded through the files service. | | `Sequences` | A reference to a sequence stored in CDF. | We support declaring all of these base and reference types as **lists**. For example, to store a list of file references: `files: [File]` You can specify whether the property is **nullable**, **immutable**, and provide a **default value**. Marking a property as immutable means that its value cannot be changed after it is set, although it is possible to toggle this setting. The full specification of a required string property can look like this: ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} name: myStringProperty description: A string property nullable: false immutable: true defaultValue: foo type: type: text list: false ``` When creating your containers, it is important to consider how they will be queried. Identify the properties likely to be used to filter and sort on when querying, and create indexes or composite indexes to support your queries. Since indexes can represent additional update overhead when data is mutated or ingested, make sure the indexes you create are useful and necessary with testing and performance validation. It is possible to set `nullable` to false on an existing property. New null values will be rejected, but existing null values remain. To check for existing null values, fetch the container and check the state fields of the property (see [Background validation](#background-validation)). ### Indexes Well-designed indexes speed up data access, they support constraints such as uniqueness, and efficient cursoring with custom sort operations. Data modeling supports up to 10 indexes per container. An index belongs to the container and is not a flag on a property. When you're laying out your physical schema, it's important to remember that you can only build indexes using properties from the same container. Indexes cannot be built from properties hosted in different containers. We support two index types: * `btree`: Use a btree index on primitive base types for efficient lookups and range scans. You can set btree indexes to be `cursorable`, and enable efficient cursoring with custom sorts. Set btree indexes to `bySpace`, indicating that they'll include the `node.space` property as a prefix on the index. * `inverted`: Use an inverted index for list-type properties to enable efficient searching for values that appear within the list. After an index is created through an API call, the service will start building the index in the background. To check the progress, fetch the container and check the `state` field of the index (see [Background validation](#background-validation)). #### Size bounds on btree indexes B-tree indexes are less effective for very large property values. Large indexed values can exceed datastore limits and cause data ingestion to fail. Large pre-existing values can cause new indexes to fail to build. To prevent ingestion failures and guard against broken B-tree indexes, you should limit the size of indexed properties. You can bound the property size in two ways: * `maxListSize` limits the number of elements in a list property. The limit can be set as high as 2000. To be included in a B-tree index, the limit must be at most 300 or 600, depending on the property type. * `maxTextSize` limits the number of UTF-8-encoded bytes in a text property. The limit can be set as high as 128k. To be included in a B-tree index, the limit mustn't exceed 2400 bytes. If a btree index has several indexed properties, the combined limits on all the properties must not exceed 2400 bytes. You must set `maxListSize` to include a property in a new B-tree index. We currently allow properties without `maxTextSize` to be included in B-tree indexes, but this will be disallowed in the future. Clean up existing data that exceeds size bounds before creating B-tree indexes. Oversized data can break new indexes and stop them from improving query performance. ### Constraints Use constraints to restrict the values that can be stored by a property, or in a container. Constraints ensure that the data has integrity, and reflects the real world. We support up to 10 constraints per container. Currently, we support two constraint types: * `uniqueness`: Ensures that the values of a property or a set of properties are unique within the container. Set a uniqueness constraint to `bySpace`, indicating that the uniqueness will apply per space. * `requires`: Points to another container, and requires that the instance has data in that other container used to populate this container. ### Background validation Creating a new constraint or index is an expensive operation that requires scanning through all existing data. Therefore, the creation endpoints return immediately without waiting for the new constraint or index to be fully valid. You won't know whether the constraint or index is fully built and valid without checking back later. To track the progress of this background work, constraints, indexes and properties returned from the service have read-only state fields, which describe the progress of the background work. State fields have three possible values: * `"current"`: The constraint is validated, or the index is built. No problems were detected. The index is ready for use. * `"failed"`: Data exists that violates the constraint, or prevents the index from being built. A failed index can't be used to speed up queries. * `"pending"`: The background work is still ongoing. Check back later. Constraints are still enforced on new data even if they are in the `failed` state. Some constraints are implicitly defined as fields on the property object, instead of being constraint objects in their own right. To track the state of these constraints, properties returned from the service have a `constraintState` field, which contains three state fields: * `nullability`: Tracks the state of the nullability constraint when setting `nullable` to `false`. * `maxListSize`: Tracks the state of the size bound constraint on list properties when setting `maxListSize` to a non-null value. * `maxTextSize`: Tracks the state of the size bound constraint on text properties when setting `maxTextSize` to a non-null value. These state fields are `null` if the property does not have the corresponding constraint. The service does not automatically revalidate or rebuild a constraint or index after it reaches the `failed` state. After you resolve the problem that caused the failure, for example by removing the pre-existing data that violated a given constraint, you can manually request a revalidation by using one of three endpoints: * [`indexes/retry`](/api-reference/concepts/20230101/containers): Retry indexes that failed to build. * [`constraints/retry`](/api-reference/concepts/20230101/containers): Revalidate requires or uniqueness constraints that had invalid pre-existing data. * [`properties/retry`](/api-reference/concepts/20230101/containers): Revalidate nullability or size-bound constraints on properties that had invalid pre-existing data. These endpoints trigger a new scan through all relevant data, so use them sparingly. ### Example container definition This example defines two containers, `Equipment` and `Pump`: * It sets `usedFor` to `node` on both containers to allow them to be populated for nodes, not edges. * The `btree` index on `Equipment.manufacturer` enables efficient sorting/filtering on the `manufacturer` property. The index is cursorable and lets you efficiently cursor through equipment nodes when sorting on `manufacturer`. * The `requires` constraint on `Pump` ensures that any node with data in the `Pump` container also has data in the `Equipment` container. ```yaml title="Pump-Equipment definition" theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} - space: equipment externalId: Equipment usedFor: node properties: - manufacturer: type: type: text list: false nullable: false indexes: manufacturer: type: btree properties: - manufacturer cursorable: True - space: equipment externalId: Pump usedFor: node properties: - maxPressure: type: type: float64 list: false nullable: false constraints: requireEquipment: constraintType: requires require: space: equipment externalId: Equipment ``` ## Views Use **views** to create logical **schemas** to consume and populate a graph tailored for specific use cases. Like containers, views contain a group of properties. You define the views by either mapping container properties, or by creating connection properties to express the expected relationships in the graph. You query data through your defined views. Data is not queried directly from the containers. ### Mapped properties Views let you **map properties** from different containers in a "flat" object and rename or alias properties. For example, this view creates a flat object with the properties `manufacturer` and `maxPressure` from the `Equipment` and `Pump` containers. It also renames the `manufacturer` property to `producer`: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph TB classDef ViewClass fill:#8250df,stroke:#24292f,stroke-width:2px,color:#fff classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Equipment[["Equipment
manufacturer (string)"]] PropertyManufacturer([manufacturer]) Pump[["Pump
maxPressure (float)"]] PropertyPressure([maxPressure]) BasicPump[["BasicPump
producer (string)
maxPressure (float)"]] Equipment:::ContainerClass --has-property--> PropertyManufacturer Pump:::ContainerClass --has-property--> PropertyPressure BasicPump:::ViewClass --maps--> PropertyManufacturer BasicPump --maps--> PropertyPressure ``` You can use the view to populate a node with data from both the `Equipment` and `Pump` containers at the same time, and query for the properties when retrieving the nodes. ### Connection properties Connection properties let you describe that you expect certain direct relations or edges to exist between nodes in the graph. When this metadata is persisted, you can retrieve related data when consuming instances through a particular view. #### Reverse direct relation Defining a reverse direct relation property between two data types is useful for describing the connections that exist in the graph. In the example below the Equipment has a direct relation to a Manufacturer. Since the relation property is part of the Equipment properties, the manufacturer view can describe the reverse direct relation. ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph TB classDef ViewClass fill:#8250df,stroke:#24292f,stroke-width:2px,color:#fff classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Manufacturer[["Manufacturer
name (string)"]] ManufacturerView[["Manufacturer
name
equipment"]] Equipment[["Equipment
name
manufacturer (Manufacturer)"]] EquipmentView[["Equipment
name
manufacturer"]] ManufacturerView:::ViewClass --maps--> Manufacturer:::ContainerClass EquipmentView:::ViewClass --maps--> Equipment:::ContainerClass ManufacturerView --reverse direct relation--> EquipmentView EquipmentView --direct relation--> ManufacturerView ``` * `connectionType`: Either `single_reverse_direct_relation` or `multi_reverse_direct_relation`, depending on if you expect a single relation or multiple objects. * `name`: The name of the property. * `description`: The description of the property. * `through`: The source and identifier of the direct relation property. * `source`: The source definition of the expected data. * `targetsList` (read-only): Whether the reverse direct relation targets a *list* of direct relations or not. The `single_reverse_direct_relation` doesn't ensure that **only** a single instance is connected. If you want to ensure only a single instance can be connected you need to ensure that the Direct Relation property has a uniqueness constraint. As reverse direct relations traverse the graph, it is highly recommended that there is a b-tree index on the direct relation property. ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} equipments: name: 'Equipments' description: 'All equipments made by the manufacturer' connectionType: multi_reverse_direct_relation through: identifier: manufacturer source: type: view space: vendor_schema externalId: Equipment version: 1 source: type: view space: vendor_schema externalId: Equipment version: 1 ``` #### Edges You can also define a connection using an edge. For example, you can express that you expect nodes with data in `BasicPump` to have `flows-to` edges to `Valve` nodes. ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} %%{init: {'theme':'base', 'themeVariables': { 'lineColor':'#8b949e'}}}%% graph TB classDef ViewClass fill:#8250df,stroke:#24292f,stroke-width:2px,color:#fff classDef ContainerClass fill:#1a7f37,stroke:#24292f,stroke-width:2px,color:#fff Equipment[["Equipment
manufacturer (string)"]] PropertyValves([valves]) Pump[["Pump
maxPressure (float)"]] Valve[["Valve
state (string)"]] BasicPump[["BasicPump
producer (string)
maxPressure (float)
valves: [BasicValve]"]] BasicValve[["BasicValve
producer (string)
state (string)"]] BasicPump:::ViewClass --has-connection--> PropertyValves BasicPump --maps--> Pump:::ContainerClass BasicPump --maps--> Equipment:::ContainerClass BasicValve:::ViewClass --maps--> Valve:::ContainerClass BasicValve --maps--> Equipment PropertyValves -.-> BasicValve PropertyValves -.-> flowsTo subgraph types flowsTo((flows-to)) end ``` The example above encodes that nodes with data in `BasicPump` can have `flows-to` edges to nodes with data in `BasicValve`. You can describe this with these fields in a connection property: * `connectionType`: Supports `single_edge_connection` or `multi_edge_connection` (default). * `type`: the fully qualified external ID of the node representing the edge type. * `source`: a reference to the view which you can view the node in the other end through. * `direction`: the direction to traverse the edge (inwards/outwards). * `edgeSource`: an optional reference to a view. A Pump/Valve would look like this: ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} type: space: types externalId: flows-to source: space: equipment externalId: BasicValve version: v1 direction: outwards ``` ### Implementing other views Views can implement other views and inherit their properties. This is useful when you want to create a view that combines the properties of multiple other views. For example: ```yaml wrap theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} - space: equipment externalId: Equipment version: v1 properties: manufacturer: container: space: equipment externalId: Equipment containerPropertyIdentifier: manufacturer - space: equipment externalId: Pump version: v1 implements: # <-- Declares that the view implements the Equipment view - space: equipment externalId: Equipment version: v1 properties: maxPressure: container: space: equipment externalId: Pump containerPropertyIdentifier: maxPressure ``` The effective properties of the `Pump` view are now `manufacturer` *and* `maxPressure`. The effective properties of a view are resolved at query time. We do not allow breaking changes to views, but if a view implemented by another view is deleted, the inherited properties will be removed from the implementing view. This could break clients if it removes any required properties. #### Implemented property conflicts and precedence If you, for example, have four views; A, B, C, and D, each with a single property with the following implements graph, you can see the effective properties on the right. ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} graph TD subgraph effective properties EA("A(a,b,c,d)") --> EB("B(b,c,d)") EB --> EC("C(c)") EB --> ED("D(d)") end subgraph properties A("A(a)") --> B("B(b)") B --> C("C(c)") B --> D("D(d)") end ``` If you introduce conflicting property identifiers in this graph, they are resolved by sorting the `implements` graph topologically. The order beneath a node is determined by the order of the implements array, where later entries are preferred. If `B implements [C, D]`, the order of precedence is `A, B, D, C`. If `B implements [D, C]`, the order is `A, B, C, D`. In these examples `B implements [C, D]`: ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} graph TD subgraph effective properties EA("A(a,b,c,d,D.x)") --> EB("B(b,c,d,D.x)") EB --> EC("C(c,C.x)") EB --> ED("D(d,D.x)") end subgraph properties A("A(a)") --> B("B(b)") B --> C("C(c,x)") B --> D("D(d,x)") end ``` ```mermaid theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} graph TD subgraph effective properties EA("A(a,b,c,d,A.x)") --> EB("B(b,c,d,C.x)") EB --> EC("C(c,C.x)") EB --> ED("D(d)") end subgraph properties A("A(a, x)") --> B("B(b)") B --> C("C(c,x)") B --> D("D(d)") end ``` ### View versioning Views are versioned, and you can not introduce breaking changes without changing the version. See [view versioning](#view-versioning) and [data model changes](#data-model-changes) sections for information about data model and view changes that force a version update. You can adapt the versioning scheme to your needs. If you don't have a preference, we recommend using an integer (whole numbers). Start your version scheme at 1, and increment the version number by one for each new version. You can use decimals — 1.1, 1.2, and 2.0 — if you want higher version granularity. In this example, the **first digit** typically increments each time you make a **significant** or a **breaking** change. Breaking changes could require changes to your application's business logic. The **fractional number** — 2nd and 3rd digits — increments when you make **minor** changes that doesn't break your API. These kinds of changes don't typically break the logic required to use the API (and models). [Semantic versioning](https://semver.org) is a widely used versioning scheme. All version increments to a data model will break, as non-breaking changes are allowed without changing the version. You can choose to implement semantic versioning, but doing so requires close attention to the reusability factor designed for CDF data modeling. The semantic versioning scheme is not designed for data modeling but for software development. The `a.b.c` versioning scheme in semantic versioning specifies that both changes in `a` and `b` are breaking changes. This leaves only `c` for non-breaking changes. In CDF, we allow certain changes to the data model without forcing you to increment the version number for the view or data model. For example, adding a new data type to an existing data model isn't a breaking change. However, in semantic versioning, this would be a breaking change. ### View filters All views have a filter field that lets you filter the nodes that are included when querying the view. For most higher-level query endpoints in DMS, the filters are applied automatically. For advanced endpoints, you have to apply the filters manually. If no filter is specified, the default `hasData` filter is applied on the list of views specified when querying. Learn more about `hasData` filters in the [querying](/cdf/dm/dm_concepts/dm_querying) article. ### Equipment example view This example illustrates a view definition for equipment: ```yaml wrap theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} - space: equipment externalId: BasicEquipment properties: producer: container: space: equipment externalId: Equipment containerPropertyIdentifier: manufacturer - space: equipment externalId: BasicValve version: v1 # Since this only maps properties in the Equipment view, we can't rely on hasData filtering. # We add a custom filter to make sure we only include nodes of the correct type. filter: equals: property: ['node', 'type'] value: { 'space': 'types', 'externalId': 'valve' } implements: # Inherit the properties from the BasicEquipment view - space: equipment externalId: BasicEquipment - space: equipment externalId: BasicPump version: v1 implements: # Inherit the properties from the BasicEquipment view - space: equipment externalId: BasicEquipment properties: maxPressure: container: space: equipment externalId: Pump containerPropertyIdentifier: maxPressure valves: type: # The edge type to traverse space: types externalId: flows-to source: # The view to view the other node in space: equipment externalId: BasicValve version: v1 direction: outwards ``` ### Caveats * When a property mapped by a view is modified or deleted, the `lastUpdatedTime` of the view will *not* be updated. * It's important to inspect which views are affected by the deletion of container. When a container is deleted, its properties are removed from any views that map the container. This can break clients if the properties are required. ## Polymorphism in views You can achieve polymorphism for views in two ways: * [Using `implements` and the implicit view `hasData` filter](#using-implements-and-implicit-view-hasdata-filtering). * [Using explicit view filters on `type`](#using-explicit-view-filters-on-type). In the sections below, `BasicPump` and `BasicValve` are subtypes of `BasicEquipment`. ### Using `implements` and implicit view `hasData` filtering When the `BasicPump` view implements the `BasicEquipment` view, the default filter on `BasicPump` is a `hasData` filter across the underlying containers: `Pump` and `Equipment`. If you filter using the `BasicEquipment` view, you'll get anything with data in the `Equipment` container. If you filter using the `BasicPump` view, you'll get anything with data in both the `Equipment` *and* the `Pump` containers. This approach to polymorphism resembles **structural** subtyping: "*if it looks like a duck and quacks like a duck, it's a duck.*" This breaks down in some cases. For example, if a view only has connection properties, there are no backing containers to apply the `hasData` filtering on. In this case, you can [use the `type` property and view filters](#using-explicit-view-filters-on-type) - see the next section. ### Using explicit view filters on `type` If, for example, you associate all pump nodes with the type `[types, pump]`, you can use a view filter to only include nodes with that type: ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} filter: equals: property: ['node', 'type'] value: { 'space': 'types', 'externalId': 'pump' } ``` To list all equipment nodes, you must explicitly include the subtypes of the view in your filter: ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} filter: in: property: ['node', 'type'] values: [ { 'space': 'types', 'externalId': 'pump' }, { 'space': 'types', 'externalId': 'valve' }, ] ``` This approach to polymorphism resembles **nominal** subtyping: "*if it's a duck, it's a duck.*" ## Data models Use **data models** to **group views** that belong together for a purpose. For example, you might define a `EquipmentInspection` data model containing a `BasicValve` and a `BasicPump` view. ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}} space: equipment externalId: EquipmentInspection version: v1 views: - space: equipment externalId: BasicPump version: v1 - space: equipment externalId: BasicValve version: v1 ``` ## Impact of changes to views and data models The tables in this section describe the impact of changes to a container, a view, or a data model. Changes that force a visible change for a consumer of data from a view or a data model are considered **breaking changes**, and require a version change to be submitted to the service. You can change the version for a data model, or view even if a version change is *not* required. This operation should be considered a controlled switch for any consumer of data from the configuration. ### Container changes Containers are not versioned like views and data models, but you can still evolve them. The possible operations on a container fall into one of three categories: * Non-breaking and allowed * Breaking but allowed * Breaking and disallowed To perform a disallowed operation on a container, use an expand-and-contract pattern: 1. Create a new container with the desired spec. 2. Re-ingest the property data into the new container for the relevant instances. 3. Create new view versions that map to these containers, and, optionally, new data model versions to include the new views if necessary. 4. When the old views (still mapping the old containers) are no longer in use, delete the old containers and views. After deleting, recreating, re-ingesting or creating a parallel container, you need to recreate any existing view-to-container mapping(s). In the table below, the "*Not allowed*" description means that the operation can't be safely implemented in the underlying data store. | Operation | Breaking change | Allowed | Description | | ------------------------------------------------------ | --------------- | ------- | -------------------------------------------------------------------------------------------------------------------------- | | Change name. | No | Yes | Changes metadata. | | Change description. | No | Yes | Changes metadata. | | Change `usedFor`. | N/A | No | Not allowed. | | Add property. | No | Yes | When the property identifier isn't in use. | | Delete property. | N/A | No | Not allowed. | | Add `requires` or `check` constraint. | No | Yes | Identifier must be unique and constraint will apply to any new values (not pre-existing ones). | | Add `unique` constraint. | N/A | Yes | Can only be included during the initial creation of the container. | | Change constraint. | N/A | No | Not allowed. | | Delete constraint. | No | Yes | Allowed. | | Add new index. | No | Yes | Allowed. | | Delete index. | No | Yes | Allowed. | | Change index. | N/A | No | Not allowed. | | Change property: `nullable` > `non-nullable`. | Yes | Yes | Changing to non-nullable may break ingestion clients, but represents an acceptable breakage. This shouldn't be unexpected. | | Change property: `non-nullable` > `nullable`. | N/A | No | Not allowed. | | Change property: `autoIncrement`. | N/A | No | Not allowed. | | Change property: `defaultValue`. | No | Yes | Changes are applied to new values added. Previously existing values remain the same. | | Change property: `description`. | No | Yes | Changes metadata meant for human consumption. | | Change property: `name`. | No | Yes | Changes metadata meant for human consumption. | | Change property: `type`. | N/A | No | Not allowed. | | Change (text) property: `list` state. | N/A | No | Not allowed. | | Change (text) property: `collation`. | N/A | No | Not allowed. | | Change (primitive) property: `list` state. | N/A | No | Not allowed. | | Change (direct relation) property: target `container`. | N/A | No | Not allowed. | | No change. | No | Yes | Allows idempotent updates. | ### View changes Operations and the impact of the operation for data modeling views. | Operation | View | Property | Relation | Direct relation | Breaking change | Description | | --------------------------------------------- | ---- | -------- | -------- | --------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Change name. | Yes | Yes | Yes | Yes | No | Changes metadata. | | Change description. | Yes | Yes | Yes | Yes | No | Changes metadata. | | Change filter. | Yes | Yes | Yes | Yes | No | Changes the result of the view, but doesn't change the form of it. | | Change implements. | N/A | N/A | N/A | N/A | Yes | May break clients. | | Change version. | N/A | N/A | N/A | N/A | Yes | A version can be bumped even if there are no changes (allows alignment of version numbers for different combinations of views). | | Add nullable property. | Yes | Yes | N/A | N/A | No | Safe change as long as there's no collision with other inherited properties, and it doesn't introduce a new mapped container. Still safe as long as the type is the same and the property doesn't change from `nullable` to `required`. | | Add non-nullable. | Yes | Yes | N/A | N/A | Yes | Breaking since a client may depend on its existence. | | Delete property. | Yes | N/A | N/A | N/A | Yes | | | Change property type. | N/A | Yes | N/A | N/A | Yes | Version change to not break consumers. | | Change container reference for base property. | N/A | Yes | N/A | N/A | No | Permit remapping consumers to new data, from other containers, without forcing a version change for consumers. | | Change source hint. | N/A | N/A | Yes | Yes | Yes | Equivalent of changing the property type. | | Change type of relation. | N/A | N/A | Yes | Yes | Yes | As long as the source type remains the same (not changed). | | Change direction of relation. | N/A | N/A | Yes | Yes | Yes | As long as the source type remains the same (not changed). | | Change source of relation. | N/A | N/A | Yes | N/A | Yes | As long as the source type remains the same (not changed). | | No change. | N/A | N/A | N/A | N/A | No | Support idempotent updates. | ### Data model changes Operations and the impact of the operation for data models in the data modeling service. | Operation | Breaking change | Allowed | Description | | ------------------- | --------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | Change name. | No | Yes | Changes metadata. | | Change description. | No | Yes | Changes metadata. | | Add a view. | No | Yes | Non-breaking if the new view doesn't conflict with any existing views in the data model. A conflict is a view using the same external ID. | | Remove a view. | Yes | Yes | Breaking since a client may depend on the existence of the data model. | | Replace a view. | Yes | Yes | When updating the view identified by the (`space`, `externalId`, `version`). Typically done to update the version of a view used by the data model. | | Change version. | Yes | Yes | A version bump without other changes is still a version bump. | | Change space. | N/A | No | Not supported. |