Cognite MQTT extractor

The Cognite MQTT extractor is a generic extractor that extracts data over the MQTT protocol, connects to an MQTT broker, and subscribes to data on one or more topics.

The MQTT extractor is hosted by Cognite, so you don't have to download and install anything. To deploy the extractor:

1. You need hostedextractors:READ and hostedextractors:WRITE capabilities to create an MQTT extractor.

2. Make sure the extractor has the following access capabilities in a Cognite Data Fusion (CDF) project: timeseries:READ, timeseries:WRITE for writing datapoints and time series, events:READ, events:WRITE, assets:READ for writing events, raw:READ, raw:WRITE for writing raw rows, or datamodelinstances:write for data models.

   dica

   You can use OpenID Connect and your existing identity provider (IdP) framework to manage access to CDF data securely. For more information, see setup and administration for extractors.

3. Navigate to Data management > Integrate > Extractors.

4. Locate the Cognite MQTT extractor and select Set up extractor.

About MQTT

MQTT is a generic message transport protocol agnostic to the message payloads, which can be on any format. The Cognite MQTT extractor supports a number of pre-defined message formats, and you can define your own.

Messages are published on topics. A topic can be any string of text, but usually it is a hierarchical structure, separated by forward slashes, for example:

company/location3/machine1/pressure

Clients can subscribe to messages by subscribing to topic filters. A topic filter can be a topic written verbatim, but it also allows for wildcards such that one subscription can yield data from multiple topics. MQTT supports single-level and multi-level wildcards.

A single-level wildcard is denoted by a + and matches any text between two slashes. For example, if you have several topics like these:

company/location3/machine1/pressure
company/location3/machine2/pressure
company/location3/machine3/pressure

You can subscribe to all of them by subscribing to this topic filter:

company/location3/+/pressure

A multi-level wildcard is denoted by a #, and matches any text. It has to be used at the end of the topic filter. For example, if you have several topics like the following:

company/location3/machine1/pressure
company/location3/machine2/pressure
company/location3/machine1/temperature

You can subscribe to all of them by subscribing to this topic filter:

company/location3/#

Message formats

See Custom data formats for hosted extractors for how to write your own custom mapping of MQTT messages. Custom formats used with MQTT jobs will receive an input argument containing the message as JSON, and a context argument containing the topic, for example

{
	"topic": "company/location3/machine/pressure"
}

The topic given in the context will be the actual topic the message arrived on, not the topic filter, so it can be used to distinguish identical messages arriving on different topics.