Configure simulator connectors

The connectors act as translators between your simulator’s specific commands and CDF, facilitating bidirectional data flow while operating as a client making outbound https-connections only.

Before you start

Infrastructure setup

A Windows virtual or physical machine with storage space for model files.
Simulation software installed with valid licenses.
Windows administrator access for service configuration.

Authentication setup

Registered service application in your Identity Provider (IdP).
Client secret for authentication.
Properly configured access groups in your IdP.
Dedicated data set in CDF for storing simulator data.
Appropriate CDF group access capabilities.

Network configuration

Firewall allowing outbound traffic to https://{cluster}.cognitedata.com.
TLS certificate validation settings.
Proxy configuration (if applicable).

Configuration settings

To configure a simulator connector, you must edit the configuration file. The file is in YAML format, and you’ll find configuration file examples listed under the config folder in the directory where the connector is installed.

Don’t base your configuration on the config.example.yml file. Instead, use config.minimal.yml as your base and copy the parts you need from config.example.yml.

Configure the file setup

Download simulator connectors from Data fusion > Integrate > Extractors and connectors.
Create config.yml in the connector’s installation directory. Start with config.minimal.yml as a baseline.
Configure authentication values using your IdP credentials.

Set up a local service account

Create a dedicated Windows service/user account on the connector machine.
Set read and write access for the service account to the connector installation folder.

Start the Windows service

Configure the Windows service to use the local dedicated account.
Start the Windows service.
Monitor startup using Windows Event Viewer.
Verify the connector registration under Integrations in the Simulators tool in the CDF user interface or by listing simulator integrations through the Cognite API.

Run the connector

When the connector starts, it reads the configuration file, authenticates with CDF, registers itself as a simulator integration, and starts processing models and routines.
When you’ve configured the connector, it registers itself and the associated simulator in CDF, and you can create models and routines.

For on-premises connectors, customers are responsible for maintaining connector uptime. Connectors are designed to operate as always-on services to ensure consistent compute availability for users.

Using values from environment variables

In the configuration file, values wrapped in ${} are replaced with environment variables with that name. For example, ${COGNITE_PROJECT} will be replaced with the value of the environment variable called COGNITE_PROJECT. The configuration file also contains the global parameter version, which holds the version of the configuration schema used in the configuration file. This document describes version 1 of the configuration schema.

Configuration sections

Connector

This section contains information on how the current connector appears in Simulators.

Parameter	Description	Required
connector.data-set-id	Enter the data set ID to read the model files and simulator routines from. Result inputs and outputs will be written to this data set.	Yes
connector.name-prefix	Enter the connector name that is registered on the API.	No
connector.add-machine-name-suffix	When set to `true` (default), the connector name is composed of the prefix `name-prefix@vm-host-name` followed by the machine name.	No
connector.simulation-run-tolerance	The connector will time out simulation runs that are older than this value (in seconds).	No
connector.fetch-runs-interval	The connector will fetch simulation runs from CDF at this interval (in seconds).	No
connector.status-interval	The connector will update its heartbeat in CDF at this interval (in seconds).	No
connector.scheduler-update-interval	The connector will check if scheduled simulations should be triggered at this interval (in seconds)	No
connector.simulation-run-load-balancing-enabled	When set to `true`, the connector participates in Simulation run load balancing (preview), and CDF assigns queued runs across healthy connectors for higher throughput. The default value is `false`, which preserves earlier behavior: runs stay tied to your connector.	No
connector.simulation-run-poll-limit	The maximum number of simulation runs the connector gets assigned at a time. When `connector.simulation-run-load-balancing-enabled` is `true` (preview), queued runs and runs assigned to this connector count together toward this limit. The default value is `10`.	No
connector.model-library.model-parsing-timeout	The maximum time, in seconds, within which the connector must finish parsing a model revision when `connector.simulation-run-load-balancing-enabled` is `true`. If parsing isn’t complete within this time, the connector marks the revision as `failure`. The default value is `3600` (1 hour).	No

Logger

Log entries are either Fatal, Error, Warning, Information, Debug, Verbose, in order of decreasing importance. Each level covers the ones of higher importance.

Parameter	Description
console	Configuration for logging to the console.
console.level	Set the minimum level of log events to write to the console. Set this parameter to enable console logging.
console.stderr-level	Log events at this level or above are redirected to standard error.
file	Configuration for logging to a rotating log file.
file.level	Set the minimum level of log events to write to file.
file.path	Enter the path to the files to be logged. For instance, if this is set to `logs/log.txt`, logs on the form `logs/log[date].txt` will be created, depending on `rolling-interval`.
file.retention-limit	Set the maximum number of log files that are kept in the log folder.
file.rolling-interval	Enter a rolling interval for log files as `day` or `hour`. The default value is `day`.
remote	Configuration for logging to CDF.
remote.enabled	Insert boolean of whether to push logs to CDF or not. The default value is `true`.
remote.level	Set the minimum level of log events to push to CDF. Valid values are `Error`, `Warning`, `Information` and `Debug`.

Cognite

This is the configuration of the connection to CDF.

Parameter	Description
host	Insert the CDF service URL. The default value is `https://api.cognitedata.com`.
project	Enter the CDF project. This is a required parameter.
idp-authentication	The configuration for authenticating to CDF.
idp-authentication.authority	The authority used with `tenant` to authenticate to Azure tenants. Use `token-url` if you’re connecting using a non-azure IdP. The default value is `https://login.microsoftonline.com`.
idp-authentication.tenant	Enter the Azure tenant used with `authority`.
idp-authentication.token-url	Insert the URL used to obtain service tokens, used for non-azure IdPs.
idp-authentication.client-id	Enter the service principal client ID.
idp-authentication.secret	Enter the service principal client secret.
idp-authentication.resource	Insert an optional resource parameter to pass along with token requests.
idp-authentication.scopes	Enter a list of scopes to pass along with the request. This will typically need to contain `[host]/.default`.
idp-authentication.audience	Insert an optional audience parameter to pass along with token requests.
idp-authentication.min-ttl	Set the requested minimum time-to-live in seconds for the token.
idp-authentication.certificate	Configuration for authenticating using a client certificate.
idp-authentication.certificate.authority-url	Insert the certificate authority URL.
idp-authentication.certificate.path	Enter the path to the `.pem` or `.pfx` certificate to be used for authentication.
idp-authentication.certificate.password	Enter the certificate password.
max-data-points-upload-queue-size	Set the maximum number of cached data points before they are uploaded to CDF. The default value is `1000000`.
cdf-retries	Set the number of automatic retries on requests to CDF.
cdf-retries.timeout	Set a timeout in milliseconds for each try. The default value is `80000`.
cdf-retries.max-retries	Set the maximum number of retries. A value less than 0 retries forever.
cdf-retries.max-delay	Set the maximum delay between each try in milliseconds. The base delay is calculated according to `125 * 2 ^ retry` milliseconds. If this is less than 0, there’s no upper limit. The default value is `5000`.
cdf-chunking	The configuration for chunking on requests to CDF. Note that increasing these may cause requests to fail, due to limits in the API. Read the API documentation before making these higher than their current value.
cdf-chunking.time-series	Set the maximum number of time series per get/create time series request.
cdf-chunking.data-point-time-series	Set the maximum number of time series per data point create request.
cdf-chunking.data-points	Set the maximum number of data points per data point create request.
cdf-throttling	Set how requests to CDF should be throttled.
cdf-throttling.time-series	Set the maximum number of parallel requests per time series operation. The default value is `20`.
cdf-throttling.data-points	Set the maximum number of parallel requests per datapoints operation. The default value is `10`.
sdk-logging	The configuration for logging of requests from the SDK.
sdk-logging.disable	Set this to `true` to disable logging of requests from the SDK. This is by default enabled.
sdk-logging.level	Set the log level to log messages from the SDK. The default value is `debug`.
sdk-logging.format	Set the log message format. The default value is `CDF ({Message}): {HttpMethod} {Url} - {Elapsed} ms`
nan-replacement	Replacement for `NaN` values when writing to CDF. The default value is none, meaning these values are removed.
extraction-pipeline.external-id	The configuration for associating this connector with an extraction pipeline. This is used for monitoring and remote configuration.
certificates	The configuration for special handling of SSL certificates. This should not be considered as a permanent solution to certificate problems.
certificates.accept-all	Accept all remote SSL certificates even if verification fails. Note: This introduces a risk of man-in-the-middle attacks.
certificates.allow-list	List of certificate thumbprints to automatically accept. This is a smaller risk than accepting all certificates.

State store

This section includes the configuration parameters for storing state in a local database.

Parameter	Description
location	Enter the path to the database file containing the state store.
database	Select which type of database to use. Only `LiteDb` is currently supported. The default value is `None`.

Maintain connectors

Connectors might require significant disk space for caching model files stored in CDF
Connectors don’t have auto-update features, so regular manual updates are necessary to get bug fixes and new features
The underlying simulation software may also require periodic updates
When troubleshooting problems, check the connector logs in the installation directory and Windows Event Logs

​Before you start

​Configuration settings

​Using values from environment variables

​Configuration sections

​Connector

​Logger

​Cognite

​State store

​Maintain connectors

Before you start

Configuration settings

Using values from environment variables

Configuration sections

Connector

Logger

Cognite

State store

Maintain connectors