Set up the OPC UA extractor

The Cognite OPC UA extractor is distributed as a Windows standard alone executable file, a Windows service, a Linux service, or a Docker container. Follow the steps below to set up the extractor.

NOTE

The steps involve using the configuration tool, which makes many calls to the OPC UA server. If you're not sure the server can handle the load, you can configure the extractor manually.

1. Check the hardware and server requirements for the extractor.

2. Download the installer/installation files for the extractor from Cognite Data Fusion (CDF).

3. Install the extractor:

   • Windows: Run the installer.
   • Linux: Extract the contents of the installation zip file to the desired location.
   NOTE

   If writing to the location requires administrator rights, the extractor must be run as admin or granted access in another way. In general, we recommend that you don't run the extractor as an admin.

4. Run the configuration tool:

   OpcuaExtractor tool -n -e [host] -u [username] -p [password] --auto-accept -o config/config.yml

   This runs the configuration tool without a configuration file and connects to the host using the specified username and password (optional). The tool outputs the resulting configuration to config/config.yml. For more details, see The configuration tool steps.

   Note that the extractor by default expects to find the configuration file in a local folder called config, so if you are running the extractor from the command line with the default installation, it should be run from the install folder, so bin\OpcuaExtractor.exe ....

   Warning

   When setting up an extractor against a new server, you should run the configuration tool to detect potential server issues and ensure a successful start to the configuration. If the configuration tool fails, the server may be non-compliant, have insufficient computing power, or scale poorly. You may still be able to run the extractor, but this will require manual configuration and debugging. If you choose not to run the configuration tool, Cognite won't offer any support.

5. Edit config/config.yml to fit your requirements.

   The most important elements are setting a suitable ID prefix and adding pushers, metrics, and logging. This might also involve disabling some features that the configuration tool discovered or adding ones that it didn't.

   1. Optionally, edit the opc.ua.extractor.Config.xml file used for configuring the OPC UA SDK.

      For most users, the default settings are suitable, but in some cases, you may need to change memory limits or similar.

6. Start the extractor with just OpcuaExtractor, or opcua-extractor if installed on Linux.

   This starts the extractor with the default settings, loading the configuration from config/config.yml.

The configuration tool steps

This section lists each action the configuration tool takes when testing the server.

In general, the configuration tool is designed to build on the existing configuration and can update an existing configuration file. Usually, it's ok to let it run with no configuration.

1. Get endpoints: Attempts to treat the given endpointUrl as a discovery server and look for endpoints there. If more secure options are available, it might suggest connecting to these. This should be cheap and quick.
2. Identify Browse limits: Issues have been found with some servers that return too few results with certain settings for the number of nodes browsed at a time or the number of desired results. This section browses part of the node tree several times, trying different parts of the hierarchy and different configurations. It may also try to browse the server hierarchy if the main node tree is too small. If the number of nodes is very large, the smallest chunks are skipped. The root of browsing is, by default, the objects folder, but you can set this using RootNode.
3. Read custom types: Looks through the data type hierarchy, and identifies any likely candidates for custom numeric data types, that you want to map to double in destination systems.
4. Identify Read limits: Tries to find a suitable limit for the Read service, reading attributes for as many nodes as possible at a time. Typically, it starts at 1000, then 100 and 10 at a time.
5. Identify data-type settings: Using the retrieved attributes, looks for arrays and strings that you want to push to CDF, then find suitable settings for the relevant configuration options.
6. Get Subscription chunk sizes: Tries to subscribe to as many nodes as possible at a time, again starting at 1000 and going down. It will also listen and see if it receives any data on any of the subscriptions.
7. Get History Read config: Tries to read history from nodes with Historizing set. If no such nodes exist, this step is skipped. The extractor tries to read the first and last parts of history and estimate the number of points in each. If the number is sufficiently large, the configuration tool will suggest enabling Backfill.
8. Get Event config: This consists of several steps:
   1. Look through the event type hierarchy and try to find custom events.
   2. Look for GeneratesEvent references between nodes.
   3. Look at the EventNotifier property to supply information about which nodes are allowed to emit events. If both GeneratesEvent and EventNotifiers are used on the server, the intersection of the two are used to find emitters.
   4. Check if the auditing property on the server is set. If it's set, then enable auditing.
9. Get namespace map: Parses the namespace map to suggest a suitable shortening of each namespace to create compact external IDs.
10. Output a configuration file.

Run as a Docker container

Use a Docker container to run the OPC UA extractor on Mac OS or Linux systems. The docker container is available on dockerhub as cognite/opcua-extractor-net.

Tip

Consider sharing a folder if a deployment uses more than one file, for example, logs and local state files.

The Docker container comes with three volumes:

• config contains the configuration file. This is optional if the extractor is set up with remote configuration.
• logs contains the log files, if enabled.
• certificates contains the application and server certificates. This volume must be configured.

Here's a docker run statement example of how to run the extractor:

docker run -v "$(pwd)/config:/config" -v "$(pwd)/certificates:/certificates" cognite/opcua-extractor-net:2.10.3