The PI extractor can send performance metrics for remote monitoring and debugging. You can configure the PI extractor to use Prometheus. If you've enabled the metrics section, the extractor uploads metrics to the configured pushgateway/server. If you're using a pushgateway, all the metrics with descriptions will be listed under the configured job name.
Prefix metric names
The metric names are prefixed with the metric source:
dotnetis for metrics on .NET GC and memory.
processis for metrics about the running process, such as the number of threads, CPU time, and memory.
pushis for metrics related to the pushgateway, such as last time pushed and last time failed.
cognite_sdkis for metrics produced by the Cognite .NET SDK. This is mostly related to the calls to the CDF API.
extractor_utilsis for metrics produced by the Extractor Utils library. This is related to the number and duration of calls to individual endpoints in CDF and metrics on state storage.
pi_extractoris for metrics specific to the operation of the PI extractor.
|pi_extractor_details||Gauge||Information about the version of the running extractor. The metrics contain the |
|pi_extractor_start||Gauge||The UNIX timestamp when the extractor was last started. |
|pi_extractor_timeseries||Gauge||The total number of time series being processed by the extractor.|
|pi_extractor_timeseries_states||Gauge||The total number of time series per extraction state. |
|pi_extractor_heartbeat_time||Gauge||The UNIX timestamp of the last heartbeat from the extractor. |
|pi_extractor_pi_connection_start||Gauge||The UNIX timestamp of the last connection attempt to the PI server. |
|pi_extractor_pi_connection_time||Gauge||The elapsed time in milliseconds for the current connection to the PI server. |
|`pi_extractor_pi_connections||Gauge||The number of times the extractor has connected to the PI server. |
Internal errors on the extractor or on the PI server will cause the extractor to reconnect to the server, increasing the value of this gauge. If the PI extractor has been running since startup without reconnecting to the PI server, the value should be 1.
|pi_extractor_pi_queries||Summary||The number and duration of queries to the PI server. This include queries to the PI Data Archive (frontfill, backfill).|
Sample count: number of queries.
Sample sum: Sum of the duration of all queries
|pi_extractor_pi_query_points||Counter||The total number of data points (AF values) returned in queries to the PI Data Archive (frontfill, backfill). |
There are two labels:
|pi_extractor_pi_data_pipe_events||Counter||The total number of events received from the PI Data Pipe (stream). |
There are three labels:
The total number of data points per (action, status, type).
(410.000 update events containing good data point updates)
|pi_extractor_pi_data_loss||Summary||The number and duration of data loss incidents observed in the PI Data Pipe (stream) |
Sample count: Number of data loss incidents observed.
Sample sum: Sum of the duration of all incidents.
|pi_extractor_event_queue_size||Gauge||The number of events fetched from the PI Data Pipe and put in queues for processing. |
(There are 100k numeric events in the queue to be processed by the extractor)
|pi_extractor_frontfill||Gauge||The maximum and minimum timestamps for the time series undergoing frontfill. The |
|pi_extractor_backfill_goal_time||Gauge||The timestamp of the current backfill goal. The backfiller works in steps. This metric indicates how far back in time the backfiller will fetch data points for the current step. Can be used as a measure backfiller progress.|
|pi_extractor_pusher_data_points||Counter||The total number of data points per action handled by the extractor before inserting into CDF. |
(there are 100k points to be added in CDF).To know the number of points that were actually uploaded to CDF, use extractor_utils_cdf_datapoints
|pi_extractor_stream_iterations||Counter||The number of stream iterations. During one iteration, the extractor streamer will dequeue events from the extractor event queues, parse the events, and upload the data points to CDF.|
(150 iterations since the extractor started).
|pi_extractor_stream_iteration_star||Gauge||The start timestamp of the current stream iteration. Can be used to calculate the streaming latency.|
|pi_extractor_streamer_data_points||Counter||The number of data points per type being handled by the streamer. |
(100k new data points handled by the streamer so far)
|pi_extractor_streamer_oldest_data_point||Gauge||The age/time in seconds of the oldest (historical) data point received in the current stream iteration. This could be used as an indication of how far back the historical updates occurred, in case some updates are lost and a refill is required. |
(Oldest historical data point in the current streaming iteration is 2 minutes old)
|extractor_utils_cdf_timeseries_requests||Summary||A summary describing the number and duration of time series requests to CDF.|
|extractor_utils_cdf_datapoint_requests||Summary||A summary describing the number and duration of data point requests to CDF.|
|extractor_utils_cdf_datapoints||Counter||The total number of data points pushed to CDF by the utils.|
|extractor_utils_cdf_invalid_data_points||Counter||The total number of data points skipped due to bad timestamp or value. CDF requires data points to be within a certain time and value range.|
|cognite_sdk_fetch_inc||Counter||The total number of POST/GET/… actions performed|
|cognite_sdk_fetch_error_inc||Counter||The total number of errors on actions.|
|cognite_sdk_fetch_retry_inc||Counter||The total number of retries on actions.|
|cognite_sdk_decode_error_inc||Counter||The total number of data errors, meaning invalid data was received from CDF.|
|cognite_sdk_fetch_latency_update||Gauge||The measured latency in milliseconds on actions performed.|
|dotnet_total_memory_bytes||Gauge||The number of memory bytes of allocated memory|
|dotnet_collection_count_total||Counter||The number of garbage collection (GC) calls, grouped by generation.|
|process_cpu_seconds_total||Counter||The total number of CPU seconds used by the process.|
|process_num_threads||Gauge||The number of current threads in use by the process.|
|process_open_handles||Gauge||The number of current open handles.|
|process_open_fds||Gauge||The number of current open file descriptors.|
|process_private_memory_bytes||Gauge||The number of current private memory bytes.|
|process_resident_memory_bytes||Gauge||The number of current resident memory bytes.|
|process_virtual_memory_bytes||Gauge||The number of current virtual memory.|
|process_working_set_bytes||Gauge||The number of memory bytes for current working set.|
|process_start_time_seconds||Gauge||The start time in UNIX time seconds.|
|push_time_seconds||Gauge||The time in UNIX time seconds data was last pushed to prometheus pushgateway.|