Configure the Studio for Petrel extractor
To configure the Studio for Petrel extractor, you must create a configuration file. This file must be in YAML format. The configuration file is split into sections, each represented by a top-level entry in the YAML format.
You can use the sample minimal configuration file as a starting point for your configuration settings.
You can also use substitutions with environment variables in the configuration files. The values wrapped in ${}
are replaced with environment variables with that name. For example, ${COGNITE_PROJECT}
will be replaced with the value of the environment variable called COGNITE_PROJECT
.
`idp-authentication`:
project: ${COGNITE_PROJECT}
idp-authentication:
tenant: ${COGNITE_TENANT_ID}
client-id: ${COGNITE_CLIENT_ID}
secret: ${COGNITE_CLIENT_SECRET}
scopes:
- ${COGNITE_SCOPE}
Cognite
Include the cognite
section to configure which CDF project the extractor will load data into and how to connect to the project. This section is mandatory and should always contain the project and authentication configuration.
Parameter | Description |
---|---|
cdf-file-storage | Set the cloud blob storage engine that will store Petrel data objects. Valid options are azure and google . |
cdf-base-url | Insert the base URL of the CDF project. The default value is <https://api.cognitedata.com >. |
cdf-project | Insert the CDF project name you want to ingest data into. |
cdf-oidc-authority | Insert the credentials for authenticating to CDF using an external identity provider. You must enter either an API key or use IdP authentication. |
cdf-oidc-client-id | Enter the client ID from the IdP. |
cdf-oidc-secret | Enter the client secret from the IdP. |
cdf-oidc-tenant | Enter the Azure tenant. |
cdf-oidc-scopes | List the scopes. |
cdf-raw-database | Enter the CDF RAW database name that will store Studio for Petrel objects state store. If the database doesn't already exist, the extractor creates a database. |
data-set-id | Enter the data set ID for the CDF Files. Set to 0 or no value if you don't want to add a data set ID. |
state-db | Select which database to store the extractor state. If you don't use this parameter, the extractor uses an in-memory state store. |
api-version-cache | Before the extractor serializes a record from Studio for Petrel, a call goes to the API to verify if the object has been loaded. This cache holds the last state for quicker restarts. The default value is state\api-version-cache.dat" |
Cache
Include the cache
section to enable or disable caching of data to disk. Use these parameters to avoid intermittent failures.
Parameter | Description |
---|---|
enable-cache | Set to true to enable caching of data to disk and avoid intermittent failures. |
cache-location | Set the path to store cached objects. You can specify this as If the remaining disk space is less than 10 GB, caching is automatically turned off. You need Modify permissions on the folder. |
Logging
Include the logging
section to set up logging to a standard output, such as a terminal window.
Parameter | Description |
---|---|
log-to-console | Set to true to log to a console. This parameter is used if you're running the extractor from a terminal, such as Powershell. |
log-to-file | Set to true to log to a file. |
log-filename | Add a prefix string in the name for the generated log files. This is an optional parameter. |
retention | Specify the maximum number of days the log files are retained in the log folder. |
level | Select the verbosity level for logging. Valid options are verbose , debug , information , warning , error , fatal . The default value is information . |