Skip to main content

Configure the PI replace utility

To configure the PI replace utility, you must create a configuration file. The file must be in YAML format.

In addition to the sections listed below, you must include these PI extractor configuration sections:

Minimal YAML configuration file

The YAML settings below contain valid PI replace utility configurations, but not all configuration options are used.

version: 2cognite:    project: ${COGNITE_PROJECT}    idp-authentication:        tenant: ${COGNITE_TENANT_ID}        client-id: ${COGNITE_CLIENT_ID}        secret: ${COGNITE_CLIENT_SECRET}        scopes:          - ${COGNITE_SCOPE}time-series:    external-id-prefix: "pi:"pi:    host: ${PI_HOST}    username: ${PI_USER}    password: ${PI_PASS}metrics:    push-gateways:        - host: "http://localhost:9091"          job: "pi-replace-job"state-store:    database: LiteDb    location: "PiReplaceStateStore.db"logger:    console:        level: "information"replace:    start-time: "2020-02-20 10:05:00"    end-time: "2020-03-20 10:05:00"    save-progress-interval: 10sdata-quality:    check-data-quality: true    report-file-name: "report"    ignore-positive-difference: true

Configure the PI replace utility

Replace

ParameterDescription
start-timeRun the PI replace utility starting from this time. You can specify the time using an ISO date string with one of these formats:
2020-02-20, 2020-02-20 10:05:00 or 2020-02-20 10:05:00.321.

You can also specify the time as a time span using one of these formats:
360:10:00:00 (UTC now - 360 days and 10 hours), 360:10:00:00.000, 10:00:00 (UTC now - 10 hours), or 10:00:00.000.

The default value is the minimum CDF timestamp: 1971-01-01 00:00:00
end-timeRun the PI replace utility ending at this time. You can specify the time using an ISO date string or a time span. See start-time for the allowed formats.

The default value is UTC now.
startup-delayInsert the delay in seconds before the PI replace utility starts. This delay does not affect start and end times.

The default value is 0.
delete-data-pointsIf you set this to true, the data points in the range are deleted from CDF before inserting data points from PI.

The default value is false.
throttlingEnter the number of time series to replace concurrently. The default value is 10.
step-hoursReplace time series in iterations going back in time in steps of this many hours. The default will replace in steps of 24 hours. Set this to zero or less to disable replacing in iterations (replace the whole time range in a single iteration).
save-progress-intervalEnter an interval in seconds to save the PI replace progress to a state store. You must configure the selected storage in the State store section.

The default value is 10.
include-prefixes, include-tags, include-patternsInclude time series based on these lists of tags (PI Point names), prefixes, and patterns. These parameters are similar to the filters used by the PI extractor.

The default value is no value.
exclude-prefixes, exclude-tags, exclude-patternsExclude time series based on these lists of tags, prefixes, and patterns. These parameters are similar to the filters used by the PI extractor.

The default value is no value.
dry-runRun the PI replace utility in dry-run mode. The replace utility compares the data points in CDF with the data points in PI, but doesn't upload any data points to CDF. The default value is false.
timeoutEnter a timeout in seconds for each replace iteration for a time series, for instance, to make sure a single iteration does not take too long, forcing PI replace to stop.

The default value is -1 (disabled).

Data quality

Include the data_quality section to compare the data points from PI with the data points in CDF to log and report any differences in the number of data points and the values of data points. If you've included the metrics section, the data quality metrics are sent to the configured Prometheus pushgateway. PI replace creates a CSV file with a summary by the end of the run.

ParameterDescription
check-data-qualitySet to true to enable data quality check. The replace utility compares data points from PI with data points in CDF and logs and reports differences. Data points are only replaced in CDF if differences are found. The default value is false.
report-file-nameInsert the name of the CSV file used to write the quality check reports. Leave this parameter empty if you don't want to save the report to a file. The default value is null.
ignore-positive-differenceIf CDF has more data points than PI, and you've set this parameter to true, the data points from CDF are not replaced with the data points from PI. The default value is false.

Events

You can configure the Cognite PI extractor to create CDF events for data loss and reconnection incidents. These incidents may cause data points to be missing in CDF. Include the events section to fetch these events from CDF at startup and replace the data points for the time range when these events occurred.

Note

If you configure this section, the PI replace utility ignores the start-time and end-time parameters in the replace section.

ParameterDescription
sourceInsert the name of the event source. This name must match the event source name configured in the PI extractor. The default value is ““.
external-id-prefixFetch only events with this external ID prefix. This setting should match the external ID configured in the PI extractor. The default value is ““.
event-start-timeFetch events from CDF with a start time higher than or equal to this time. The allowed string formats are the same as the start-time parameter in the replace section. For example, 5:00:00:00 will fetch events starting 5 days ago. The default value is ““.
event-end-timeFetch events from CDF with an end time less than or equal to this time. For example, 00:20:00 will fetch events ending 20 minutes ago. The default value is ““.
expand-beforeSubtract this time span from the start time of the replace range (expand the range back in time). See the allowed time span formats in the start-time parameter in the replace section. The default value is ““.
expand-afterAdd this time span to the end time of the replace range (expand the range forward in time). The default value is ““.

State store

Include the state-store and configure the save_progress_interval parameter in the replace section to store the PI replace progress. This is useful for long-running operations where a crash or restart may occur. PI replace will resume from where it stopped instead of starting again.

ParameterDescription
databaseEnter the database you want to use. LiteDb creates a local file with the database. Raw will store data in CDF RAW. The default value is none.
locationInsert the database file path for LiteDb or database name for Raw. The default value is ““.