cdf migrate, that assists in this process.
The migration process is divided into a prerequisite phase, a preparation phase, and three distinct data migration phases. Each phase builds on the previous one, so complete them in order. You do not need to complete all phases — it depends on the requirements of your CDF project and which asset-centric resources and CDF features you currently use.
This guide gives you an overview of the steps to follow. Each step has a dedicated guide, and throughout, this guide uses a wind farm example introduced in the Kelmarsh example.
Prerequisites
Use a development CDF project
Migration from asset-centric to data modeling is a significant change to your CDF project. Try the migration process in a development CDF project before running it in production. See Testing migration of production data for how to snapshot your production data and restore it in a development project.Data model and resource view mapping
Before starting, ensure the following are in place:- A target data model that extends the CogniteCore model. You can create this with Neat.
- (Optional but strongly recommended) Mappings from your asset, event, time series, and file structures to the data model. These must be created manually.
Preparation
Deploy the CogniteMigration data model
The Cognite Toolkit tracks lineage from asset-centric to data modeling-based resources in a data model calledCogniteMigration. This lineage is required for the application-level migration and access management phases. Deploy the data model by running:
Instance spaces and source systems
Before starting asset-centric resource migrations, migrate data set resources and createCogniteSourceSystem nodes:
CogniteSourceSystem nodes from the source field on assets, events, and files. They also store lineage information in the CogniteMigration data model, which later migration phases require. Each command outputs the configuration for the created instance spaces and source systems to a tmp/ directory. Move these files into a Cognite Toolkit module and govern them going forward.
For more information, see the data sets migration guide and the source systems migration guide.
Asset-centric resource migrations
This phase migrates asset, event, time series, and file resources to instances (nodes) in data modeling. These resources underpin Canvas, Charts, annotations, InField, and 3D, so they must be migrated first. Migrate assets before all other resources, as assets are the anchor for all dependent resources.Migrate assets
Migrate events
Migrate files and time series
Migrate annotations
Once assets and files have been migrated, migrate annotations:Migrating sequences
Sequences are not currently supported in the migration plugin.Application-level migrations
This phase migrates Canvas, Charts, InField, and 3D models.Migrate Canvas
Migrate annotations before Canvas. Canvas looks up annotations for display, and annotations migrated after Canvas will not appear.
Migrate Charts
Migrate 3D models
Migrate 360 images
Migrate InField
Get a template for new InField configurations:Migrate relationships
Relationships are not currently supported in the migration plugin.Access management migration
Update access groups to use data modeling instance spaces and the schema space instead of asset-centric data sets and resource types. This is currently a manual process.Manual tasks
After completing all migration phases, update the data ingestion and extraction processes to write data to the data modeling-based CDF project:- Update extractors to write data to the correct instance spaces and data model concepts.
- Update transformations to write to your new data model instead of the asset-centric model.
- Update dashboards and visualizations to use the new data model concepts.