Work with F5 Beacon¶
F5 Beacon tracks the health and status of different application components, defines application structure and allows users to create customized dashboards to get a global view of their applications.
The main entities in Beacon are:
- Source - endpoints sending telemetry data to Beacon; for example, BIG-IP as a source sends stats (status, metrics, etc.) on virtual servers
- Application - Beacon’s representation of an application, depicted as a tree structure composed of components
- Component - a single entity in the application tree structure with health derived from one or more metrics and/or dependencies on other components
- Monitor - a synthetic test to monitor uptime and performance for HTTP endpoints and DNS lookups
- Widget - a visualization module that utilizes Beacon data to provide insight on the application portfolio
- Dashboard - a customizable grouping of widgets
- Insight - Pertinent information about the application portfolio. Insights are categorized as cost, operational, or security. Custom insights are created by customers via API or the portal. Built-in insights are provided by F5.
- Metric - A time data series that is defined by source, list of filters and a single field
To utilize Beacon, users configure endpoints (i.e. sources) to ingest telemetry data to Beacon, create monitors, onboard applications, create insights and create dashboards to help capture all these insights.
Beacon API and Postman collection¶
F5 Beacon offers a single pane of glass, where telemetry data from different sources can be fused together to provide actionable insights. Telemetry data can be ingested from different sources, leveraging built-in integrations such as F5 BIG-IP, F5 BIG-IQ and many more.
For the latest set of integrations refer to Configuration > Integrations.
Adding a new source¶
Navigate to Configuration.
Navigate to the Integrations tab and click the link on each tile to view the instructions. For the purposes of this documentation, we will focus on F5 BIG-IP integration using Telemetry Streaming as an example. Refer to the Beacon Getting Started Guide for a more in-depth tutorial on how to onboard other sources.
Follow the instructions provided and create a token for use on BIG-IP.
On the Tokens tab, click Create. Provide a name and optional description and click Create Token.
Click the Show button and copy the access token for use in the Telemetry Streaming declaration.
After following the instructions for configuring Telemetry Streaming, navigate to the Sources tab and see the newly added BIG-IP source.
- It can take up to a minute for a device to be processed and appear in the list of sources.
- If the device still does not appear, validate the device has outbound connectivity to Beacon and/or check the Telemetry Streaming log.
Onboard an application¶
Applications in Beacon are composed of one or more components. For example, an application may be composed of an API service and data processing services. Components can be nested in a tree structure to represent the dependencies that they have on each other.
Applications can be created using the declarative API or using the portal. Applications consist of a tree structure, starting with the application at the root. The application will include components as dependencies, which themselves may have dependencies. For any component in the tree, one or more health sources can be defined. The application can also include metadata in the form of key/value labels.
The telemetry data that sources send to Beacon can be used to define health for a component. This is referred to as a health source. The health for each component is derived from the combination of associated health sources and dependencies on other components. When creating applications, the portal makes it easy to select source data used to define health.
In the above example, metrics for healthy BIG-IP virtual servers are attached as health sources for the Data process API, API Gateway and Database components. A metric for an unhealthy BIG-IP virtual server is attached as a health source to the Backend API component.
Navigate to Beacon via the Cloud Services navigation menu.
On the Application Landscape page, click Create.
The Create Application page will appear. Initially, a side panel will be shown to configure the root component. Provide a name, plus optional description and labels if desired. When complete, click Save & Continue to move to the next section.
The Component Health Settings section on the Create Application page allows you to configure a metric health condition to the root component directly. If not configured, its health will be derived solely from its children components.
Finish creating the application by clicking Save. You can add additional components to the application as desired.
Display application on the map¶
An application can be placed on the map by adding the following labels (case sensitive) on the root component of an application.
location: this label is used as the display text for the location (e.g. Seattle)
latitude: this label is used to capture the latitude coordinates (e.g. 47.60357)
longitude: this label is used to capture the longitude coordinates (e.g. -122.32945)
Filter by location using the map¶
View application details¶
Click on the row for an application to see a slide panel that shows health history, associated insights and other details for the application.
To see a more detailed view of an application, click on the application name in the List View or one of the headers in the slide panel. The detailed view provides multiple tabs showing various aspects of the application. For components with health metrics, graphs of the metrics can be viewed on the Application Map tab.
The Health & Events tab displays health history and related events for an application.
From the Application Map tab, applications can also be edited, by toggling to Edit Mode.
When creating or editing an application, the JSON Configuration tab shows the equivalent JSON for the application. The JSON is editable and suitable for use with the Declare API.
Be sure to save any changes before navigating to another tab.
Set component health settings¶
An application component in Beacon has a health status. A health status defines the operational level of the component. In the case of a root component, it defines the entire application health status. Health status values are:
- Healthy (value 2) - application or component is operational
- Warning (value 3) - application or component is in a degraded state
- Critical (value 4) - application or component is not operational
- Disabled (value 5) - application or component configured to not be operational
- None (value 1) - application or component does not have health information
A component’s health status is calculated from two sources:
- The health status of the component’s dependencies
- Metric health conditions associated with the component
If the component has multiple dependencies, you must define how to aggregate the dependencies’ health status values to calculate the component’s health status value. This is done by toggling the ‘aggregated health status of this component and dependent components’ field with values:
- Best Of - Set health status based on the best health status between this component and dependent components.
- Worst Of - Set health status based on the worst health status between this component and dependent components.
To configure a component’s health status using metric values, you can configure a Metric Health Condition. A Metric Health Condition is a rule to evaluate/deduce health. A condition must be comprised of a metric to evaluate and there are two types of metrics a condition can use:
- Built-In Health Metric - Metrics with a “healthStatus” field, which already defines a valid health value without needing thresholds/limits.
- Generic Health Metrics - Metrics that are ingested into Beacon and require thresholds/limits that define how to translate values into a health status. For example, CPU over 80% could define a warning status.
For Built-In Health Metrics, there are a handful of source types currently supported:
- bigip-virtual - health defined by a BIG-IP virtual server
- bigiq-app - health defined by a BIG-IQ application
- bigiq-app-svc - health defined by a BIG-IQ application service
- monitor - health defined by a Beacon HTTP monitor
- dns-monitor - health defined by a Beacon DNS monitor
- beacon-health-source - health defined by anything feeding data to the Beacon generic ingestion API
- uptime-robot - health defined by an Uptime Robot monitor
A Metric Health Condition that is using a Generic Health Metric must configure a critical and/or warning threshold, comparison operator, time range in minutes and aggregation function to translate metric values to a health status. A component can have multiple conditions associated with it.
In the Component Health Settings section of the Create Application page, click the add button.
Choose a metric, an evaluation period and fill both the critical and/or warning conditions and define the metric health used when no metric data is found.
If the value is above 90, it sets the condition health to critical, and if the value is above 70, it sets the condition health to warning. Otherwise, it’s defined as healthy.
If a component has only a single metric health condition and no dependencies, the metric health condition will match the component health status. If a component has multiple metric health conditions, each providing a different health status, the component must aggregate those values to a single component health status.
This is done by toggling the ‘aggregated metric health conditions’ field with values:
- Best Of - Set health status based on the best health status between the metric health conditions
- Worst Of - Set health status based on the worst health status between the metric health conditions
To summarize, component health is calculated in the following manner:
- Computing the health status value for each Metric Health Condition configured on the component.
- Aggregate those conditions to a single metric health status
- Compute the component health status by aggregating the metric health status with health status of the component’s dependencies
Diagram that describes that process:
Currently, monitors originate from our US East region only. In future releases, we are looking to enhance this feature so that you can select additional regions. Stay tuned!
HTTP monitors track the status of an HTTP endpoint. The result can be used as a health source for application components. Each test will result in the following metrics being collected in Beacon. These metrics can be then queried within the portal when creating insights.
- DNS Lookup (ms)
- TCP Connection (ms)
- TLS Handshake (ms)
- Server Processing (ms)
- Content Transfer (ms)
- HTTP Request Time (ms)
- HTTP Status Code
- Health Status (AVAILABLE, WARNING, CRITICAL)
- Health Reason (e.g. Monitor check failed with status code: 502)
- SSL Certificate Expiry Days Left (certExpiryDaysLeft metric)
- SSL Error Status (certError metric)
- The resulting HTTP status codes are mapped to health values as follows:
- < 400 is AVAILABLE
- >= 400 is CRITICAL
View HTTP monitor latency metrics
When a monitor is added to the root of an application, the application slide panel shows latency metrics for the monitor. Click on the row for an application to show its slide panel.
Additional SSL checks apply to HTTP monitors by default when the URL starts with https://. SSL checks result in a source type called ssl-check which includes various metrics, most notably the SSL score (finalScore metric). The SSL score (ranging from 0 to 100) is based on the validation of your SSL certificate, while accounting for protocol, key exchange and cipher support in your server configuration. A zero in any category will bring the overall score to zero. Any warning or critical findings can impact the monitor health.
Findings that result in critical health include:
- Certificate revoked (via CRL or OCSP check)
- Most situations in which SSLv2 is available
- Bad ciphers are offered (null, anonymous, export)
- Inability to validate chain of trust
- Signature algorithm is MD2/MD4/MD5
- Cert key size less than 110 bits
- RSA certificate uses exponent of 1
- Self-signed certificate
- Issuer not trusted anymore (WoSign/StartCom)
- Cert expired
- Private key is known via check of pwnedkeys.com DB
- Secure renegotiation is not supported
- Vulnerable to:
- CCS Injection
Findings that can result in warning health include:
- Redirects to an insecure URL
- Problems with HTTP Public Key Pinning (HPKP)
- Some scenarios where SSLv2 or SSLv3 is offered
- Client is allowed to pick cipher when it should not
- Problem with OCSP stapling
- Low cipher strength (64 Bit + DES, RC[2,4], MD5 (w/o export))
- Certificate incorrectly used for digital signatures / key encipherment / key agreement / TLS Web Server Authentication
- Issues with cert CN/SAN
- Cert expires soon
- Cert expires in greater than 10 years
- Neither CRL nor OCSP URI provided
- Insecure Client-Initiated Renegotiation
- Only SSLv3 supported
- TLS 1.3, 1.2, 1.1, 1.0 not supported
- Vulnerable to:
- POODLE SSL
DNS monitors are similar to HTTP monitors, but track the results of a DNS lookup. The result can be used as a health source for application components. Each test will result in the following metrics being collected in Beacon. These metrics can be then queried within the portal when creating insights.
- DNS Lookup (ms)
- Failure Count (number of consecutive failed tests)
- Name Server Failure Count (number of consecutive failed tests per name server)
- Health Status (AVAILABLE, CRITICAL)
- Health Reason
For details about additional capabilities not covered here, such as providing headers or transforming content and specifying the DNS record type or custom name servers, see Declare API.
The Monitors page under Configuration lists existing monitors and is used to create, modify and delete monitors.
Select HTTP or DNS under the Type dropdown and complete the form. You can also expand Set Advanced Configuration for additional configurations.
Dashboards are used to visualize the current and past state of the application portfolio. A set of widgets is provided, each with specific visualization, such as application health history. Widgets can be added to dashboards to create customized views.
The Dashboard Management page is used to create, modify and delete dashboards. The Dashboard View page is used to display dashboards.
Add a dashboard¶
To create a new dashboard:
Select Dashboard Management. A list of existing dashboards is displayed. To create a new dashboard, click Create.
On the dashboard creation page, there are two tabs, Properties and Layout.
On the Properties tab, provide a name and an optional description.
Navigate to the Layout tab and construct the widget layout by dragging widgets onto the dashboard. You can move and resize the widgets as you require.
There are six types of widgets available:
- Application Map - display the tree layout of a single specified application
- Application Current Health - display application counts grouped by current health and a specified custom label
- Application Health History - display application counts grouped by health over time
- Events - display details about application health transitions and trends over time
- Insight - display a single insight
- Insights Summary - display counts of insights grouped by category and severity
When complete, click Save Dashboard.
To view existing dashboards, select Dashboard View from the menu. Each existing dashboard is available as a separate tab. In addition to showing the widgets configured, each dashboard includes a Application Details table. The applications included in the list change based on any filtering applied. Clicking on some widgets will automatically apply filtering to the list.
To edit an existing dashboard:
- Either click the edit icon at the top right when viewing a dashboard or select Dashboard Management from the F5 Beacon dropdown menu and then click on the dashboard name.
- Editing a dashboard is similar to what was described in the Add a dashboard section.
To delete an existing dashboard:
- Navigate to Dashboard Management.
- Select the checkbox for the dashboard and click the Delete button at the top right of the grid. A popup will appear to confirm the deletion.
The Insights page within F5 Beacon is a centralized location to get insightful information about your application portfolio. Insights are divided into three categories Cost, Operations and Security.
Beacon includes two sets of insights:
- Built-in insights are provided by F5 Networks and made available within the service. Such insights include the ‘F5 Assets and Inventory’ insight detailing F5 assets acting as Beacon sources and the ‘F5 App Protection’ insight detailing security findings for BIG-IQ HTTP applications. Built-in insights are updated automatically and cannot be edited or deleted. The built-in insights appear only when relevant data exists (e.g. ‘F5 Assets and Inventory’ appears when F5 sources exist).
- Custom insights are based on analyzing any raw data stored in F5 Beacon or outside of it. These insights can be created either via the portal or API.
Create custom insights via the portal¶
Insights can also be created within the portal:
Navigate to Insights via the dropdown menu and click Create.
Select a Source Type and one Metric. Filters can also be selected to filter the results. The default metric name will be changed to the Metric field name you selected and you can click on the tab to rename. Once complete, click Run to view your custom insight.
Select a visualization type and time period to display. Click and drag within a graph or use the slider at the top of the graph to zoom in for a detailed view.
Add additional metrics to your insight. Click the + icon to add another metric and complete the menu selection following abovementioned instructions.
View multiple metrics created. Adjust visualization type and time period to display following abovementioned instructions.
To save an insight, provide a title and category at the bottom. You can optionally provide a description and one or more Associated Applications.
To edit existing insights, click the insight name from Insights.
Data aggregation in custom insights¶
Data displayed for custom insights is not the raw data, but aggregated. When viewing data for the Last 1 day, the resolution is 4 minutes, meaning data points within each 4 minute period are aggregated together. The function selected determines how the data points are aggregated. For example, for a source that generates a data point every minute, the first twelve minutes of data will be represented by three data points, each of which is an aggregation of 4 data points in the raw data.
|Function||Point 1||Point 2||Point 3|
Resolution is defined as follows:
- Last 1 day - 4 minutes
- Last 1 week - 30 minutes
- Last 1 month - 2 hours
- Last 1 quarter - 6 hours
- Last 1 year - 1 day
Associate custom insights to a Beacon application via the portal¶
You can associate the created Insights to your Beacon application within the portal:
Navigate to Application Landscape via the dropdown menu and click on the application you want to associate your insights with.
Use Edit Mode to edit the root component. Under Associated Insights, select one or more insights to associate to this application.
Use Read Only to view the insight graph within the slide panel for an application root component.
The insight title cannot be updated once the insight is created. Changing a title can be accomplished by deleting an insight and creating a new insight with the updated title.
If you ingest data into Beacon less frequently than every 4 minutes (last day interval resolution), gaps can appear with the column charts and table widgets when viewing data by last day interval.
If using metric data that isn’t recent, the metric may not appear in graphs using short time horizons. Expand the time horizon (e.g. Last 1 Month) to view the older metric data.
Create custom insights via API¶
Beacon users can generate custom insights and publish them to Beacon via the insights API.
See the Beacon GitHub account where there are detailed documentation and sample code for creating custom insights, including a Telegraf Ping plugin example that generates insight from the metrics reported by the Telegraf Ping plugin.
Beacon provides the ability to store metrics and subsequently query them via the portal or API to generate custom insights.
You can leverage many ways to ingest data into Beacon. For the purposes of this documentation, we will highlight an example using Telegraf. Telegraf is an open-source server agent that can collect metrics from multiple sources and systems.
Integrate Telegraf with Beacon¶
Telegraf has a wide range of input plugins that can gather metrics and send them to Beacon. For a detailed list, see Telegraf plugins. To get started with Telegraf, see Getting started with Telegraf and follow the setup instructions.
Select an Input plugin based on the type of metrics you want to collect. Use the HTTP plugin for the Output plugin.
# A plugin that can transmit metrics over HTTP [[outputs.http]] ## URL is the address to send metrics to url = "https://ingestion.ovr.prd.f5aas.com:50443/beacon/v1/ingest-metrics" ## Timeout for HTTP message timeout = "120s" ## HTTP method method = "POST" ## Optional TLS Config # tls_ca = "/etc/telegraf/ca.pem" # tls_cert = "/etc/telegraf/cert.pem" # tls_key = "/etc/telegraf/key.pem" ## Use TLS but skip chain & host verification insecure_skip_verify = true ## Data format to output. data_format = "influx" ## HTTP Content-Encoding for write request body content_encoding = "identity" ## Additional HTTP headers [outputs.http.headers] Content-Type = "text/plain; charset=utf-8" X-F5-Ingestion-Token = "...."
- url - the URL field points to Beacon ingestion endpoint
- tls - the TLS configuration
- outputs.http.headers - add an X-F5-Ingestion-Token header containing an Beacon source token, which is used for authentication
Once Telegraf is configured to run with the outputs.http plugin, Beacon will start receiving metrics sent by the Telegraf agent.
By default, Telegraf agents include a host tag. Beacon uses this tag to identify Telegraf sources. The value can be overridden in the Telegraf configuration via the agent.hostname parameter. If agent.omit_hostname is enabled, Beacon will not be able to identify the source (it will not appear on the Sources page).