Aggregator metrics#

Overview#

Introduction#

The Packet Inspector 1 aggregator service exposes metrics that you can scrape using Prometheus.

Note

Some metrics are not available in all versions of Aspen Mesh.

Metric endpoint configuration#

The Packet Inspector 1 aggregator metrics are available at the following endpoint.

Port	Endpoint
9106	/metrics

Metric types#

Metric types are defined by Prometheus. Learn about Prometheus metric types.

Observe metrics in Grafana using the sample dashboard#

Introduction#

If you want to observe metrics for Packet Inspector 1 in Grafana, you can import the sample Packet Inspector 1 dashboard provided with Aspen Mesh to use as a starting point for your own dashboard.

Note

The sample Packet Inspector 1 dashboard includes both aggregator and filter metrics.

Before you begin#

Before you can observe metrics for Packet Inspector 1 in Grafana, make sure the following software is installed on your cluster:

Prometheus
Grafana
Packet Inspector 1

Import the sample dashboard#

Log in to Grafana.
Click Dashboards in the left-side menu.
Click the New pop-up menu, and then choose Import.
Upload the following dashboard JSON file:
- Directory: samples/aspenmesh/dashboards (in the Aspen Mesh release directory)
- File: packet-inspector-1-dashboard.json
Click Import.

Metrics#

aspenmesh_packet_inspector_analysis_processing_ms#

Description#

This metric consists of two submetrics that let you determine the average number of milliseconds an aggregator pod spent communicating with the analysis service for a request:

aspenmesh_packet_inspector_analysis_processing_ms_sum: The total number of milliseconds the aggregator pod spent communicating with the analysis service
aspenmesh_packet_inspector_analysis_processing_ms_count: The number of requests the aggregator pod processed that weren’t dropped

Type#

Summary

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP

aspenmesh_packet_inspector_circular_buffer_bytes#

Description#

The total number of bytes currently in use by an aggregator pod’s circular buffer

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`

aspenmesh_packet_inspector_circular_buffer_capacity#

Description#

The total number of messages that could currently be stored in an aggregator pod’s circular buffer without growing

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`

aspenmesh_packet_inspector_circular_buffer_length#

Description#

The total number of messages currently stored in an aggregator pod’s circular buffer

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`

aspenmesh_packet_inspector_connections_active#

Description#

The number of active connections currently in use by an aggregator pod

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`peer`	The aggregator pod’s communication peer	- `filter`: Filters - `analysis`: The analysis service (your network analysis tool or the analysis emulator)
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP

aspenmesh_packet_inspector_message_groups#

Description#

The total number of message groups currently stored in an aggregator pod’s circular buffer. A Diameter message group consists of a request and its answer (and any retransmitted messages). An HTTP message group can contain both a request and its response or any subset.

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`

aspenmesh_packet_inspector_partial_capture_total#

Description#

The total number of message groups sent to the analysis service where some messages were dropped from the group prior to sending

Type#

Counter

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP

aspenmesh_packet_inspector_queue_channel#

Description#

The total number of requests currently in an aggregator pod’s queue channel, which temporarily holds requests received from filters

Note

This metric typically reports zero requests because this portion of the code is not a bottleneck in practice.

Type#

Gauge

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`

aspenmesh_packet_inspector_request_bytes_total#

Description#

This metric consists of two submetrics that let you determine the average number of bytes an aggregator pod sent to the analysis service or received from filters for a request:

aspenmesh_packet_inspector_request_bytes_total_sum: The total number of bytes the aggregator pod sent to the analysis service or received from filters
aspenmesh_packet_inspector_request_bytes_total_count: The number of requests the aggregator pod processed

Type#

Summary

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`peer`	The aggregator pod’s communication peer	- `filter`: Filters - `analysis`: The analysis service (your network analysis tool or the analysis emulator)
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP
`status`	The result of the processing	- `success`: Requests that were successful - `error`: Requests resulting in an error - `retrying`: Requests whose delivery was retried - `dropped`: Requests that were dropped because the circular buffer was full

Note

Only the following peer/status label-value combinations are used:

`peer`	`status`
`filter`	`success`
`analysis`	`success`
`analysis`	`error`
`analysis`	`retrying`
`analysis`	`dropped`

aspenmesh_packet_inspector_requests_total#

Description#

The total number of requests an aggregator pod sent to the analysis service or received from filters

Type#

Counter

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`peer`	The aggregator pod’s communication peer	- `filter`: Filters - `analysis`: The analysis service (your network analysis tool or the analysis emulator)
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP
`status`	The result of the processing	- `success`: Requests that were successful - `error`: Requests resulting in an error - `retrying`: Requests whose delivery was retried - `dropped`: Requests that were dropped because the circular buffer was full

Note

Only the following peer/status label-value combinations are used:

`peer`	`status`
`filter`	`success`
`filter`	`error`
`analysis`	`success`
`analysis`	`error`
`analysis`	`retrying`
`analysis`	`dropped`

aspenmesh_packet_inspector_total_duration_ms#

Description#

This metric consists of two submetrics that let you determine the average number of milliseconds an aggregator pod spent processing a request:

aspenmesh_packet_inspector_total_duration_ms_sum: The total number of milliseconds the aggregator pod spent processing requests
aspenmesh_packet_inspector_total_duration_ms_count: The number of requests the aggregator pod processed

Type#

Summary

Labels#

Label name	Description	Label values
`hostname`	The hostname of the aggregator pod exposing the metric	Example: `aspen-mesh-packet-inspector-g6pfk`
`peer`	The portion of the request-processing timespan	- `filter`: The time spent communicating with filters - `analysis`: The time spent communicating with the analysis service (your network analysis tool or the analysis emulator) - `combined`: The entire time spent processing requests that weren’t dropped, including the time requests spent in the aggregator pod’s circular buffer - `dropped`: The entire time spent processing dropped requests, including the time requests spent in the aggregator pod’s circular buffer
`protocol`	The protocol of the captured data	- `diameter`: Diameter - `http`: HTTP

Note

The time an aggregator pod spends communicating with filters is very low, so the value of this metric when peer is filter is frequently nearly 0 ms.
When peer is combined, the value of this metric includes the time the aggregator pod spent communicating with the analysis service. To determine the actual overhead of the aggregator pod’s processing, subtract the aspenmesh_packet_inspector_analysis_processing_ms metric.