Prerequisites
Before migrating InField, complete the following:Required capabilities
The following capabilities are required to runcdf migrate infield-configs and cdf migrate infield-data:
Migration overview
InField migration has two required phases and two optional phases. Test each phase in a development or staging project before you run it in production.Enable InField alpha flags
To migrate InField, set theinfield alpha flag in addition to migrate:
Migrate InField configurations
Run cdf migrate infield-configs
APM_Config model
and generates a configuration template (<location_name>.InfieldCDMLocationConfig.yaml) and space definitions YAML files (<space_name>.Space.yaml) for each location, written to the
tmp/ folder. It also generates location filter per location that you can
optionally deploy to enable location filtering in the Search UI.This step only reads existing InField configurations and writes templates — it makes no configuration changes to the CDF project or InField itself.Review and edit the generated configuration
dataStorage and viewMappings.dataStorage controls which space InField writes its data to:rootLocation is the root asset for the given location, and appInstanceSpace
is the space InField will write its data to.viewMappings defines which views InField should use for each resource type. By
default these point to CogniteCore (CDM) and Cognite IDM views — these can however be updated
to use custom views extending from these instead:cdf_infield/FieldObservation view — for example to support
SAP writeback — add an observation entry that points to that view:writeBack is only required (and should only be used) if your custom observation view supports sending observations to SAP. notificationsEndpointExternalId
and attachmentsEndpointExternalId reference the SAP endpoints InField sends to — they don’t affect data migration, but must be set
for SAP writeback to work once the location is deployed. See Set up the SAP writeback service
for how to configure these endpoints.Move the files to your modules/ folder
tmp/ into a custom-defined module inside your modules/
directory in the repository you manage with the Cognite Toolkit. It is up to you to decide how to structure your modules.
As an example, you could create a module called infield/ and move all your InField-related config YAML files there.Deploy the configuration
dataStorage and viewMappings point to the correct spaces and views.
Migrate InField data
Before you migrate InField data in a production CDF project, Cognite strongly recommends copying a snapshot to a development or staging project and running the migration there first. That lets you find mapping or data issues before you touch production.Copy legacy InField data to a development project
Use the Cognite Toolkit data plugin to download from the legacy InField modelcdf_apm:ApmAppData(version=v13) in production and upload to development or staging. To include edges, enable extend-download = true in cdf.toml as shown above.
Download all InField-related data:
--env-path here because these commands target different environments — downloading from
production and uploading to dev/staging — while your active cdf.toml points to only one of them.
Alternatively, you can manually replace the credentials in your .env file with the production credentials,
but take care not to mix up credentials for different environments.
After downloading the data, upload it to the development or staging environment:
cdf migrate infield-data command requires that configuration in the target environment.
Run cdf migrate infield-data
After you copy the data and deploy the configuration, run cdf migrate infield-data to write legacy InField data to the new model:
cdf migrate infield-data migrates observations to the view you configured under viewMappings.observation
in the InField location(s) that target your appInstanceSpace (see Migrate InField configurations), falling back to the default
cdf_infield/FieldObservation view if you did not configure a custom one.
If you use a custom observation view, this is a best-effort migration. It works well when your custom view is a simple extension of FieldObservation, i.e.
you have all the same properties as FieldObservation and you only added the sapStatus and notificationIdInSap fields to support SAP writeback for custom observation views.
If you renamed property identifiers or changed their types, migrate observations manually using the
--skip-observations flag and follow Migrate to custom observations. If you are unsure whether your view can
be migrated automatically, run the command without --skip-observations first and inspect the migration logs.
See Migrate to custom observations for example warnings to look for.
Verify the migration
In the development or staging project, confirm that observations, checklists, templates, and related data appear in the expected spaces and CDM views. Review the migration logs for errors before you proceed to production or optional phases.SAP writeback for custom observation views
status field served two purposes: business workflow state (for example, Draft or Completed)
and SAP send-state (for example, Sent or Not sent). The new InField observation types split this into two dedicated fields:
a dedicated business status field, and — if you need SAP writeback — two additional fields that mirror the legacy SAP-specific behavior:
cdf migrate infield-data
detects these fields on your configured view and automatically splits the legacy status value into the new status and sapStatus
fields:
sapStatus and notificationIdInSap properties, cdf migrate infield-data
still migrates the business status (as Completed or Draft, per the table above), but reports an error for every affected observation
because the SAP send-state can’t be preserved. If this happens, make sure to add the two properties to your view and run the migration again.
An example container that defines the two SAP writeback properties could look like this. pending is included because the migration
itself can now write it (for legacy Pending observations), and it’s also the state InField sets on sapStatus while a live writeback
request is in flight.
FieldObservation:
Migrate to custom observations (optional)
cdf migrate infield-data can
automatically map, migrate those observations outside the standard process. Skip this section if you use the default FieldObservation view, or if you see no warnings or errors when migrating observations.
You’ll typically discover you need this section by running cdf migrate infield-data without --skip-observations first, and finding
one or more of these warnings in the migration logs:
cdf migrate infield-data: it upserts observations by external ID, so re-migrating
updates the existing instances instead of duplicating them.
With the exception of the SAP writeback warning, if you see any of these errors or similar, it means your defined view is not supported by
the migration plugin, and will need to be handled by custom scripting. You might only need to migrate a few of the observations or some of
the properties manually however, use the generated log messages to help you identify the gap.
Here is a suggested step-by-step process to migrate the observations manually:
Download the legacy observations data
cdf_apm:ApmAppData (version=v13) data model, then the
cdf_apm:Observation(version=v5) view. You can choose to include only certain instance spaces if you want. You
do not need to include edges for this view. Finally, make sure you select csv (or parquet) as the output format.Run your custom migration script
Create a manifest file for the new observation view
Upload your custom observations data
Migrate InField source data (optional)
APM_SourceData data model and you have not already populated your IDM model.cdf migrate infield-source-data migrates the work order data stored in the legacy APM_SourceData data model — specifically maintenance orders (activities), operations, and notifications — into the cdf_idm views used by InField on CDM.
Prerequisites: Assets must be migrated and InField configurations must be deployed before running this command. The command reads destination spaces and views from the deployed InField CDM location configs, and resolves asset cross-references through the CogniteMigration model.
Required capabilities
How source and target spaces are determined
The command prompts you interactively to select a source space and a target space. The available choices are derived automatically from deployed config — you cannot specify arbitrary spaces. Source space candidates are collected from yourAPM_Config node.
Target space candidates are any space that appears as the entry in dataFilters.maintenanceOrders.instanceSpaces, dataFilters.operations.instanceSpaces, or dataFilters.notifications.instanceSpaces (these are expected to use the same space for a location) on any deployed InField CDM location config.
instanceSpaces array at runtime, so only that space qualifies as a candidate target.How destination views are determined
For each entity type, the command checksviewMappings.<type> on every deployed InField CDM location config
whose dataFilters.<type>s.instanceSpaces[0] matches the selected target space:
viewMappings entry for a given type are skipped and do not affect conflict detection.
Property mapping
These are the mappings applied out of the box. If you use custom views with different property identifiers, some of these mappings may fail to apply. Similarly, some properties are intentionally ignored out of the box, as they have no equivalent in thecdf_idm view(s). You may create a custom view to replace the cdf_idm views
if you want to bring these properties over, but for the migration script to be able to handle mapping these, the target property needs to use the exact same identifier and type as the source property.
APM_Activity → CogniteMaintenanceOrder
CogniteMaintenanceOrder: assignedTo, subActivities, functionalLocation, isInApp, resource, rootLocation, source, cloneOf, isMutated, mutatedFields, createdAt, lastUpdatedDate, isArchived.
APM_Operation → CogniteOperation
cloneOf, source, type, isInApp, createdAt, isMutated, files, lastUpdatedDate, subOrder.
APM_Notification → CogniteNotification
rootLocationExternalId, system, Country, subSystem, source.
Caveats and known limitations
- Asset references must be migrated first. If an asset has not been migrated, the asset reference is omitted from the output node and an error is written to the migration log.
- Source view variation between projects. Projects configured with
featureConfiguration.viewMappingsread from different source views than the default, which may mean your configuration is not reflective of what is described in this documentation. - Idempotent upserts. Re-running the migration after fixing errors is safe — existing nodes are updated rather than duplicated.
Migrate to a development or staging environment first
As with Migrate InField data, we recommend copying a snapshot of the source data to a development or staging project and runcdf migrate infield-source-data there before you run it in production.
Use the Cognite Toolkit data plugin to download the source data from
production and upload it to the development or staging environment.
--env-path flag specifies which credentials file to use. It is needed
here because these commands target different environments — downloading from
production and uploading to dev/staging — while your active cdf.toml
points to only one of them.APM_SourceData, adjust the values accordingly):
cdf migrate infield-source-data also needs to read the APM_Config node to determine the source space candidates, so populate
that too:
cdf migrate infield-source-data relies on the deployed InField CDM location configs to determine target
spaces and views.
Running the migration
--source-space and --target-space together, or neither.Run the migration in production
Once you have verified the migration in development or staging — with no unexpected data issues or mapping errors — run the same steps in production:- Deploy the InField configurations you reviewed and tested (if you deployed them only to development or staging earlier).
- Run
cdf migrate infield-data(with--skip-observationsif applicable). - Optionally run
cdf migrate infield-source-data.
What cdf migrate infield-configs does
When you run cdf migrate infield-configs, the Cognite Toolkit performs the following steps:
Download the legacy InField configuration
APM_Config:APM_Config(version=1) data model.Create new InField configuration templates
tmp/ folder by default. The template lookup reads the root asset from the InstanceSource view
in the CogniteMigration data model, so assets must be migrated and the root asset must be set up in InstanceSource
before you run this command.Create location filters
Write the new configuration templates to files
tmp/ folder by default. Move these files
to your modules/ folder and deploy them — the Cognite Toolkit does not deploy templates automatically
because you are expected to review and edit them first.What cdf migrate infield-data does
When you run cdf migrate infield-data, the Cognite Toolkit performs the following steps in batches:
Download nodes and edges from the legacy InField model
cdf_apm:ApmAppData(version=v13) data model that have
properties in the ConditionalAction, Observation, Checklist, MeasurementReading, Template,
Action, Condition, ChecklistItem, TemplateItem, and Schedule views.Convert legacy InField nodes to the new InField model
InstanceSource view in the CogniteMigration data model to find the corresponding assets,
time series, and files that the InField data references, and updates those references to point to the nodes in the new model.Write the new InField data to CDF
cdf_infield:InFieldOnCMD(version=v1).