Cognite Event Hub extractor

The Cognite Event Hub extractor connects to the Azure Event Hub and extracts data using the Advanced Message Queing (AMQP) protocol. Cognite hosts the Event Hub extractor, so you don't have to download or install anything.

Before you start

• Assign access capabilities to create an Event Hub extractor and for the extractor to write data points, time series, events, RAW rows and in to data models in the target CDF project.

  conseil

  You can use OpenID Connect and your existing identity provider (IdP) framework to manage access to CDF data securely. Read more.

Deploy the extractor

1. Navigate to Data management > Integrate > Extractors.

2. Locate the Cognite Event Hub extractor and select Set up extractor.

3. Get a connection string that you'll need to set up the Event Hub extractor:

   1. Locate your Event Hub in Azure.
   2. Navigate to Hub Settings > Built-in endpoints > Event Hub compatible endpoint.

Typically, it's sufficient to create a single job per Event Hub source. An Event Hub is equivalent to a single topic on other streaming platforms. You can't filter data received from an Event Hub.

conseil

When you connect to an Event Hub, you specify a ConsumerGroup. We recommend to create a new consumer group for each application that reads from your Event Hub.

Message formats

Azure Event Hub is a data streaming service on Azure that supports payloads in any format. The Cognite Event Hub extractor supports several predefined message formats. If you want to define your own custom mapping of Event Hub messages, see custom data formats for hosted extractors. Custom formats used with Event Hub jobs will receive an input argument containing the message in a JSON format.