Before you start
- Assign access capabilities to create an MQTT broker source and for the extractor to write data points, time series, events, RAW rows, and instances in data models to the target CDF project, depending on the resource types you want the extractor to ingest.
- Before you create the job, reuse or create a destination. If you use a custom message format, reuse or create a mapping as described in Custom message formats.
- The MQTT broker must be enabled for your project. Contact Cognite support to request access.
- The Cognite MQTT broker is not available for Private Link–enabled CDF projects.
This setup uses two different credentials:
- CDF API (create the source and job): use your usual CDF authentication, such as OpenID Connect (OIDC), as described in setup and administration for extractors.
- MQTT publisher (send data): use the
host,username, andpasswordreturned when you create the source.
Set up the MQTT broker
Create a source
Create a The response contains the connection details your publisher needs:
Source by sending a request to /api/v1/projects/{project}/hostedextractors/sources. Include the cdf-version: beta header. For the full request schema, see the Source API specification.Create a job
Create a Messages must be transformed into a CDF resource type. For supported formats and custom mappings, see Message formats.If the job is not
Job by sending a request to /api/v1/projects/{project}/hostedextractors/jobs. The job must reference your destination and mapping. For the full request schema, see the Job API specification.Set topicFilter to match the topics your publisher will send data on.A
200 response with connected or running in the response body confirms that you created the job.connected or running, work through the checks below.Job not connected or running
Job not connected or running
You may see one of these responses:
200withconnection_errororstartup_errorin the body400 Bad Requestor422 Unprocessable Entity
- Your authentication token is valid.
- Your request payload matches the request schema.
Connect your publisher
Configure your MQTT client to connect using the credentials from step 1:If your publisher does not connect, work through the checks below.
- Host: the
hostvalue from the response in step 1 - Port:
8883 - Username: the
authentication.usernamevalue - Password: the
authentication.passwordvalue - TLS: required. See Encryption for certificate requirements.
A success message or response state in your MQTT client confirms that your publisher connected to the broker.
Publisher not connected
Publisher not connected
Your MQTT client may show an error message or a failed connection state. Use your MQTT client documentation to identify, decode, and resolve specific error codes, then try connecting again. Also verify the following.
- The host, port, username, and password match the values from step 1.
- TLS is enabled and your device trusts the broker certificate. See Encryption.
- You saved the password from step 1 and have not reset it since.
Managing credentials
The password can be reset at any time by sending aPOST request to /api/v1/projects/{project}/hostedextractors/sources/update with the cdf-version: beta header:
Topics and topic filters
Messages are published to topics — hierarchical strings separated by forward slashes:+) and multi-level (#) wildcards.
+matches a single level.company/+/pressurematchescompany/machine1/pressureandcompany/machine2/pressure.#matches all remaining levels and must appear at the end.company/location3/#matches all topics undercompany/location3/.
context.topic. See Message formats for examples of input and context arguments.
Message formats
The Cognite MQTT broker supports pre-defined message formats and custom message formats. Custom mappings receive two arguments:input, which contains the message payload as JSON, and context, which contains the topic the message arrived on.
For example, when a publisher sends {"pressure": 42.1} on topic company/location3/machine1/pressure, the mapping receives the following values.
input (message payload):
context is the actual topic the message arrived on, not the topic filter. Use context.topic to distinguish messages arriving on different topics.