Skip to main content

Extractor metrics

The PI extractor can send performance metrics for remote monitoring and debugging. You can configure the PI extractor to use Prometheus. If you've enabled the metrics section, the extractor uploads metrics to the configured pushgateway or server. If you're using a pushgateway, all the metrics with descriptions will be listed under the configured job name.

Prefix metric names​

The metric names are prefixed with the metric source:

  • dotnet is for metrics on .NET GC and memory.

  • process is for metrics about the running process, such as the number of threads, CPU time, and memory.

  • push is for metrics related to the pushgateway, such as last time pushed and last time failed.

  • cognite_sdk is for metrics produced by the Cognite .NET SDK. This is mostly related to the calls to the CDF API.

  • extractor_utils is for metrics produced by the Extractor Utils library. This is related to the number and duration of calls to individual endpoints in CDF and metrics on state storage.

  • pi_extractor is for metrics specific to the operation of the PI extractor.

Metrics​

NameTypePurpose
pi_extractor_detailsGaugeInformation about the version of the running extractor. The metrics contain the version label with the version of the running extractor. The actual metric value can be ignored.

Example:
version="2.1.0.0"
pi_extractor_startGaugeThe UNIX timestamp when the extractor was last started.

Example:
1599495797230 (7 September 2020 16:23:17.230)
pi_extractor_timeseriesGaugeThe total number of time series being processed by the extractor.
pi_extractor_timeseries_statesGaugeThe total number of time series per extraction state.
The state label indicates whether the state is frontfill, backfill, or streaming. Example:
state="frontfill" value=13378 (13.000 time series are currently being frontfilled)
pi_extractor_pi_connection_startGaugeThe UNIX timestamp of the last connection attempt to the PI server.

Example:
1599495797590
pi_extractor_pi_connection_timeGaugeThe elapsed time in milliseconds for the current connection to the PI server.

Example:
7030683.7172 (~1.95 hours)
pi_extractor_pi_connectionsGaugeThe number of times the extractor has connected to the PI server.
Internal errors on the extractor or on the PI server will cause the extractor to reconnect to the server, increasing the value of this gauge. If the PI extractor has been running since startup without reconnecting to the PI server, the value should be 1.
pi_extractor_pi_queriesSummaryThe number and duration of queries to the PI server. This includes queries to the PI Data Archive (frontfill, backfill).

Sample count: number of queries.
Sample sum: Sum of the duration of all queries
pi_extractor_pi_query_pointsCounterThe total number of data points (AF values) returned in queries to the PI Data Archive (frontfill, backfill).

There are two labels: mode and state.
  • mode is either frontfill or backfill.
  • state is either good or bad.


    • Example:
    mode="frontfill" state="good" value=179070022 (~180 million good data points received from frontfill queries)
    pi_extractor_pi_data_pipe_eventsCounterThe total number of events received from the PI Data Pipe (stream).

    There are three labels: action, status and type.

  • action is either Add, Insert, Delete, Refresh, or Update.
  • status is either good or bad.
  • type is either AFDataPipeEvent or AFDataPipeRangeDeletedEvent (C# class type of the event)


    • The total number of data points per (action, status, type).
    Example:
    action="Update" status="good" type="AFDataPipeEvent" value=410045
    (410.000 update events containing good data point updates)
    pi_extractor_pi_data_lossSummaryThe number and duration of data loss incidents observed in the PI Data Pipe (stream)

    Sample count: Number of data loss incidents observed.
    Sample sum: Sum of the duration of all incidents.
    pi_extractor_event_queue_sizeGaugeThe number of events fetched from the PI Data Pipe and put in queues for processing.
    The name label indicates the type of the queue: double and string.

    The double queue contains data pipe events related to numeric data points.

    The string queue contains data pipe events related to string data points. Total number of events in the extractor queues.

    Example:
    name=”double” value=100000
    (There are 100k numeric events in the queue to be processed by the extractor)
    pi_extractor_frontfillGaugeThe maximum and minimum timestamps for the time series undergoing frontfill. The aggregate label indicates if it's the max or the min timestamp.
    Example:
    aggregate=”min” value=1546077427120 and aggregate=”max” value=1600256700000 (The minimum timestamp of all time series frontfilling is 29 December 2018 09:57:07.120 and the maximum is 16 September 2020 11:45:00. That is, there is about 2 year of data to be frontfilled).
    pi_extractor_backfill_goal_timeGaugeThe timestamp of the current backfill goal. The backfiller works in steps. This metric indicates how far back in time the backfiller will fetch data points for the current step. This can be used as a measure backfiller progress.

    Example:
    value=1599652198896 (This timestamp is ~ UTC 9 September 2020. If the current UTC date is 16 September 2020, then the current backfill step is fetching data for one week back).
    pi_extractor_pusher_data_pointsCounterThe total number of data points per action handled by the extractor before inserting into CDF.

    The action label specifies how the data point is being handled.

    added are points to be uploaded to CDF.

    bad are points discarded because they're marked as bad in PI.

    frontfill-skipped are data points skipped during frontfill.

    backfill-skipped are points skipped during backfill.

    Example:
    action=”added” value=100000
    (there are 100k points to be added in CDF).To know the number of points that were actually uploaded to CDF, use extractor_utils_cdf_datapoints
    pi_extractor_stream_iterationsCounterThe number of stream iterations. During one iteration, the extractor streamer will dequeue events from the extractor event queues, parse the events, and upload the data points to CDF.

    Example:
    value=150
    (150 iterations since the extractor started).
    pi_extractor_stream_iteration_starGaugeThe start timestamp of the current stream iteration. This can be used to calculate the streaming latency.

    Example:
    value=1600265491419 (16 September 2020 14:11:31). Comparing it to UTC now (or even push_time_seconds) provides an indication of how late the current iteration is.
    pi_extractor_streamer_data_pointsCounterThe number of data points per type being handled by the streamer.

    label type indicates the type of event in the PI server that generated the data point.

    new are real-time data being added to the time series (newer than the newest data point in a time series).

    historical are updates/insertion of data points in the past.

    ahead are new data points that are ahead of the newest data point in CDF because the frontfiller hasn't caught up with the real-time data points.

    Example:
    type=”new” value=100000
    (100k new data points handled by the streamer so far.)
    pi_extractor_streamer_oldest_data_pointGaugeThe age/time in seconds of the oldest (historical) data point received in the current stream iteration. This could be used as an indication of how far back the historical updates occurred, in case some updates are lost and a refill is required.
    Example:
    value=120
    (Oldest historical data point in the current streaming iteration is 2 minutes old.)
    extractor_utils_cdf_timeseries_requestsSummaryA summary describing the number and duration of time series requests to CDF.
    extractor_utils_cdf_datapoint_requestsSummaryA summary describing the number and duration of data point requests to CDF.
    extractor_utils_cdf_datapointsCounterThe total number of data points pushed to CDF by the utils.
    extractor_utils_cdf_invalid_data_pointsCounterThe total number of data points skipped due to bad timestamp or value. CDF requires data points to be within a certain time and value range.
    cognite_sdk_fetch_incCounterThe total number of POST/GET/… actions performed
    cognite_sdk_fetch_error_incCounterThe total number of errors on actions.
    cognite_sdk_fetch_retry_incCounterThe total number of retries on actions.
    cognite_sdk_decode_error_incCounterThe total number of data errors, meaning invalid data was received from CDF.
    cognite_sdk_fetch_latency_updateGaugeThe measured latency in milliseconds on actions performed.
    dotnet_total_memory_bytesGaugeThe number of memory bytes of allocated memory
    dotnet_collection_count_totalCounterThe number of garbage collection (GC) calls, grouped by generation.
    process_cpu_seconds_totalCounterThe total number of CPU seconds used by the process.
    process_num_threadsGaugeThe number of current threads in use by the process.
    process_open_handlesGaugeThe number of currently open handles.
    process_open_fdsGaugeThe number of currently open file descriptors.
    process_private_memory_bytesGaugeThe number of current private memory bytes.
    process_resident_memory_bytesGaugeThe number of current resident memory bytes.
    process_virtual_memory_bytesGaugeThe number of current virtual memory.
    process_working_set_bytesGaugeThe number of memory bytes for the current working set.
    process_start_time_secondsGaugeThe start time in UNIX time seconds.
    push_time_secondsGaugeThe time in UNIX time seconds data was last pushed to the Prometheus pushgateway.
    Last updated