Documentation Index
Fetch the complete documentation index at: https://docs.cognite.com/llms.txt
Use this file to discover all available pages before exploring further.
Before you start
- Assign access capabilities to create a hosted MQTT extractor and for the extractor to write data points, time series, events, RAW rows, and instances in data models in the target CDF project.
Deploy the extractor
Message delivery and reliability
Before you define message formats and mappings, review how data reaches CDF. Reliable delivery depends on your publishers, your broker, and the Cognite MQTT extractor, not on the extractor alone. The extractor subscribes to topics at Quality of Service level 1, which provides at-least-once delivery, and requests persistent sessions from the broker. For reliable message delivery during brief disconnections, your broker must persist messages for disconnected clients and retain session state. When the extractor reconnects, the broker can deliver unacknowledged messages. The broker applies the lower of the publisher’s QoS and the extractor’s subscription QoS. QoS 0 publishes reach the extractor at QoS 0 only, even though the extractor subscribes at QoS 1. To verify that data reaches CDF, use the job metrics in Hosted extractors overview. If the broker shows traffic but CDF does not update, work through the checks below.Broker shows messages but CDF does not update
Broker shows messages but CDF does not update
Symptoms: The broker or your publisher logs show traffic, but time series, events, or other CDF resources do not update.Checks:
- Confirm publisher QoS, broker persistence, and sessions as described earlier in this section.
- In the Hosted extractors overview, open the job and compare Source messages with Destination uploaded values and related metrics to see whether failures happen at transform or write steps.
- If you use an extraction pipeline to track ingestion, see Monitor extraction pipelines for how to interpret Topic filters status and uptime, and troubleshoot failed integrations. The same article explains troubleshooting for on-premises and hosted extractors.
Message formats
MQTT is a generic message transport protocol agnostic to the message payloads, which can be in any format. The Cognite MQTT extractor supports several pre-defined message formats. If you want to define your own custom mapping of MQTT messages, see custom data formats for hosted extractors. Custom formats used with MQTT jobs will receive aninput argument containing the message as JSON, and a context argument containing the topic, for example:
+ denotes a single-level wildcard and matches any text between two slashes. For example, if you have several topics like these:
# denotes a multi-level wildcard and matches any text. It has to be used at the end of the topic filter. For example, if you have several topics like the following:
Encryption
The Cognite MQTT extractor supports connecting to MQTT brokers with Transport Layer Security (TLS) encryption. Select Enable Transport Layer Security (TLS) on the connection when you set up the connection to the broker. Upload the root certificate to the Cognite MQTT extractor when you’re creating the connection if your broker uses a custom Certificate Authority (CA) to create and sign the certificate used for encryption. The extractor supports PEM and DER formats.Certificate versionThe Cognite MQTT extractor only supports X.509 certificates of version 3. Earlier versions will not work with the extractor.