Work with F5 Beacon

F5 Beacon tracks the health and status of different application components, defines application structure and allows users to create customized dashboards to get a global view of their applications.

The main entities in Beacon are:

  • Source - sends analytics data and stats to Beacon; for example, BIG-IP sources send stats on virtual servers
  • Application - Beacon’s representation of an application, depicted as a tree structure composed of components
  • Component - a single entity in the application tree structure with health derived from one or more metrics and/or dependencies on other components
  • Widget - a visualization module that utilizes Beacon data to provide insight on the application portfolio
  • Dashboard - a customizable grouping of widgets
  • Insight - information about the cost, security, and operational aspects of the application portfolio

To utilize Beacon, users configure sources to publish analytic data to Beacon, onboard applications and create a dashboard with widgets to gain insight on application status.

In this tutorial, we will demonstrate how to:

  1. Configure a BIG-IP source to publish analytics data to Beacon
  2. Onboard an application with health defined by data received from the BIG-IP
  3. Create a dashboard which visualizes the application portfolio

Use the F5 Beacon portal

Access the F5 Beacon portal by using the Beacon tab in the Cloud Services navigation menu.

_images/CS-Beacon.Service.png

Beacon has the following menu items:

  • Dashboard View - view configured dashboards
  • Insights View - view built-in and custom insights by category and severity
  • Application Inventory - view a list of applications with health status
  • Component Inventory - view a list of components with health status
  • Dashboard Management - create, modify and delete dashboards
  • Source Configuration - configure sources to provide application health and analytics data

Configure source

A source is any entity that sends health and analytic data to Beacon. This tutorial shows configuring a BIG-IP as a source. To configure BIG-IP to send analytics data to Beacon, we use telemetry streaming, part of the F5 Automation Toolchain.

Add a BIG-IP device source

  1. Navigate to Beacon via the Cloud Services navigation menu.

  2. Via the drop down menu, select Source Configuration.

    _images/CS-Beacon.Service-Source-Config.png
  3. Navigate to the Integrations tab and click the link to view the instructions for integrating BIG-IP.

    _images/CS-Beacon.Service-Source-Config-Create-Integration.png
  4. The instructions provide a link to create a token for use on BIG-IP. On the Tokens tab, click Create.

    _images/CS-Beacon.Service-Source-Config-Create-Token.png
  5. The Create Token side panel will appear. Provide a name and optional description and click Create Token.

    _images/CS-Beacon.Service-Source-Config-Token-Table.png
  6. Select the token in the grid to see its details. Copy the access token for use in the Telemetry Streaming declaration.

    _images/CS-Beacon.Service-Source-Config-Token-Instructions.png
  7. After following the instructions for configuring Telemetry Streaming, navigate to the Sources tab and see the newly added BIG-IP source.

Note

  1. It can take up to a minute for a device to be processed and appear in the list of sources.
  2. If the device still does not appear, validate the device has outbound connectivity to Beacon and/or check the Telemetry Streaming log.

Onboard an application

Applications in Beacon are composed of one or more components. For example, an application may be composed of an API service and data processing services. Components can be nested in a tree structure to represent the dependencies that they have on each other.

_images/CS-Beacon.Service-App-map.png

The analytics data that sources send to Beacon can be used to define health for a component. This is referred to as a health source. The health for each component is derived from the combination of associated health sources and dependencies on other components. When creating applications, the portal makes it easy to select source data used to define health.

_images/CS-Beacon.Service-App-map-health.png

In the above example, metrics for healthy BIG-IP virtual servers are attached as health sources for the Data process API, API Gateway and Database components. A metric for an unhealthy BIG-IP virtual server is attached as a health source to the Backend API component.

Authenticate against F5 Cloud Service API

To send an API request to any F5 Cloud Service API, a bearer token must be obtained, see API overviews and example for additional detail.

Specify preferred account header in a multiple accounts/divisions scenario

A user can be associated with multiple accounts or have configured divisions. In these scenarios, to disambiguate the account information, a header named X-F5aaS-Preferred-Account-Id needs to be added with the account ID value.

Account membership information is available via the following APIs.

Retrieve the user ID and primary accountID via:

GET https://api.cloudservices.f5.com/v1/svc-account/user
{
    "id": "u-aaPgo44WXX",
    "email": "user@f5.com",
    "first_name": "John",
    "last_name": "Smith",
    "phone": "+1 (555) 111-1111",
    "primary_account_id": "a-aaT6XYZa9j",
    ...

Retrieve account memberships via:

GET https://api.cloudservices.f5.com/v1/svc-account/users/<USER_ID>/memberships
{
    "memberships": [
        {
            "account_id": "a-aaSXXdAYYY2",
            "account_name": "NW - Zone",
            "role_name": "privileged-user",
            ...

        },
        {
            "account_id": "a-aahXYZLa7H",
            "account_name": "f5",
            "role_name": "privileged-user",
            ...
        },
        {
            "account_id": "a-ahhhZMXXJU",
            "account_name": "HR",
            "role_name": "owner",
            ...
        },
        {
            "account_id": "a-aaT6XYZa9j",
            "account_name": "F5 Networks",
            "role_name": "owner",
            ...
        }
    ]
}

Note

It is recommended to always set the preferred account header to make any future transition to a multiple accounts/divisions scenario easier.

Create application

The Beacon Declare API can be used to create an application. Applications consist of a tree structure, starting with the application at the root. The application will include components as dependencies, which themselves may have dependencies. For any node in the tree, one or more health sources can be defined. The application can also include metadata in the form of key/value labels.

Applications can also be created within the portal:

_images/CS-Beacon.Service-App-map-create.png

When attaching metrics to an application to define health, there are a handful of measurements currently supported:

  • bigip-virtual - health defined by a BIG-IP virtual server
  • bigiq-app - health defined by a BIG-IQ application
  • bigiq-app-svc - health defined by a BIG-IQ application service
  • monitor - health defined by a Beacon monitor
  • beacon-health-source - health defined by anything feeding data to the Beacon generic ingestion API
  • uptime-robot - health defined by an Uptime Robot monitor

Create a monitor

Monitors track the status of an HTTP endpoint. The result can be used as a health source for application components. In addition to applications, the Beacon Declare API can also be used to create a monitor. Monitors are configured with:

  • name - unique name
  • description - optional description
  • url - full URL to monitor, including scheme, host and port
  • method - HTTP method to use when polling (e.g. HEAD or GET)
  • interval - defining frequency of polling (in seconds)
  • type - one of: - HEALTH_CHECK - for directly testing endpoint availability - DELEGATE_TRANSFORM - which queries the endpoint and then forwards (via POST) the response to another endpoint (the transformUrl noted below) that is expected to generate content suitable for sending to the Beacon generic ingestion API
  • transformUrl - required for monitors of type DELEGATE_TRANSFORM
  • requestHeaders - optional headers to be sent with the request to url in the form header1:value1,header2:value2
  • transformHeaders - optional headers to be sent with the request to transformUrl in the form header1:value1,header2:value2

Dashboards

Dashboards are used to visualize the current and past state of the application portfolio. A set of widgets is provided, each with specific visualization, such as application health history. Widgets are added to dashboards to create customized views. The Dashboard Management page is used to create, modify and delete dashboards. The Dashboard View page is used to display dashboards.

Add a dashboard

To create a new dashboard:

  • Via the Beacon drop down menu, select Dashboard Management. A list of existing dashboards is displayed. To create a new dashboard, click Create.

    _images/CS-Beacon.Service-Dashboard-mgmt.png
  • On the dashboard creation page, there are two tabs, Properties and Layout.

    • On the Properties tab, provide a name and optional description.

      _images/CS-Beacon.Service-Dashboard-mgmt-layout.png
    • Navigate to the Layout tab and construct the widget layout by dragging widgets onto the dashboard. There are five types of widgets available:

      • Treemap - display application counts grouped by current health and a specified custom label
      • Application Health History - display application counts grouped by health over time
      • App map - display the tree layout of a single specified application
      • Events - display details about application health transitions and trends over time
      • Insights - display counts of insights grouped by category and severity
    • When complete, click Save Dashboard.

View dashboards

To view existing dashboards:

  1. Via the Beacon drop down menu, select Dashboard View. Each existing dashboard is available as a separate tab. In addition to showing the widgets configured, each dashboard includes a list of Application Details. The applications included in the list change based on any filtering applied. Clicking on some widgets automatically applies filtering to the list.

    _images/CS-Beacon.Service-Dashboard-view.png

Edit dashboards

To edit an existing dashboard:

  • Either click the edit icon at the top right when viewing a dashboard, or select Dashboard Management on the Beacon drop down menu and then click on the dashboard name.
  • Editing a dashboard is similar to what was described in the Add a dashboard section.

Delete dashboards

To delete an existing dashboard:

  • Via the Beacon drop down menu, select Dashboard Management.
  • Select the checkbox for the dashboard and click the delete button at the top right of the grid. A popup will appear to confirm the deletion.

Declare API

By using the declare API, you can add and modify, or remove multiple applications and monitors with a single call. The declare API path is https://api.cloudservices.f5.com/beacon/v1/declare, where you can POST a declaration of your applications and monitors.

Declare API request:

{
    "action": "deploy",
    "declaration": [{
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "AccountingApp",
                "description": "An accounting application",
                "labels": {
                    "department": "Accounting",
                    "location": "Azure"
                },
                "healthSourceSettings": {
                    "metrics": [
                        {
                            "measurementName": "bigip-virtual",
                            "tags": {
                                "name": "virtualServerName",
                                "source": "big-ip1.f5.com"
                            }
                        }
                    ]
                }
            }
        }
    ]
}

Declare has two fields:

  • action - the type of operation to apply, with valid values of “deploy” (the default), “get”, “dry-run”, and “remove”
  • declaration - an array of configuration resources

Each declaration resource contains metadata with the following:

  • version - the API version to use to create the resource

Get action - Viewing existing resources

The “get” action is used to view existing resources. It returns the full set of configured applications and monitors.

Get action - View resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "get"
}

Deploy action - Creating and updating resources

The “deploy” action is used to create resources. This action is asynchronous – the response includes a task ID that should be polled to get task progress and confirm success or failure.

When the declaration includes existing resources, the “deploy” action will update the resources.

Remove action - Deleting resources

The “remove” action is used to explicitly delete resources. Resources not included in the declaration using the “remove” action are not impacted.

Full resources definitions are allowed in the declation when using the “remove” action, though you can also include only the resource name.

Remove action - Delete resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "remove",
    "declaration": [
        {
            "application": {
                "name": "AccountingApp"
            }
        },
        {
            "application": {
                "name": "FinanceApp"
            }
        }
    ]
}

Dry-run action - Validate resources

The “dry-run” action validates the supplied resources and responds with the operation (“add” or “update”) that will occur if a subsequent “deploy” action occurs.

Response:

Declare API "dry-run" reply
{
    "action": "dry-run",
    "declaration": [{
            "metadata": {
                "version": "v1",
                "operation": "update" <<<<
            },
            "application": {
                "name": "AccountingApp"
            }
        },
        {
            "metadata": {
                "version": "v1",
                "operation": "add" <<<<
            },
            "application": {
                "name": "MarketingApp"
            }
        }
    ]
}

Metrics

Beacon provides the ability to store metrics and subsequently query them to generate insights and publish them to dashboards. To send metrics to Beacon, you can utilize Telegraf, the open-source server agent that can collect metrics from multiple sources and systems.

Setting up Telegraf with Beacon

Telegraf has a wide range of input plugins that can gather metrics and send them to Beacon. For a detailed list, see Telegraf plugins. To get started with Telegraf, see Getting started with Telegraf and follow the setup instructions.

Select an Input plugin based on the type of metrics you want to collect. Use the HTTP plugin for the Output plugin.

# A plugin that can transmit metrics over HTTP
[[outputs.http]]

  ## URL is the address to send metrics to
  url = "https://ingestion.ovr.prd.f5aas.com:50443/beacon/v1/ingest-metrics"

  ## Timeout for HTTP message
  timeout = "120s"

  ## HTTP method
  method = "POST"

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  insecure_skip_verify = true

  ## Data format to output.
  data_format = "influx"

  ## HTTP Content-Encoding for write request body
  content_encoding = "identity"

  ## Additional HTTP headers
  [outputs.http.headers]
    Content-Type = "text/plain; charset=utf-8"
    X-F5-Ingestion-Token = "...."
  • url - the URL field points to Beacon ingestion endpoint
  • tls - the TLS configuration
  • outputs.http.headers - add an X-F5-Ingestion-Token header containing an Beacon source token, which is used for authentication

Once Telegraf is configured to run with the outputs.http plugin, Beacon will start receiving metrics sent by the Telegraf agent.

Querying Beacon metrics

You can use the metrics API to query Beacon for metrics. Beacon is using InfluxDB to store metric data and as a result, the query language is InfluxQL. See https://docs.influxdata.com/influxdb/v1.7/query_language/spec/ for syntax details on how to execute a metric query.

For example, if using the Ping input plugin to publish ping results:

## Ping given url(s) and return statistics
[[inputs.ping]]
  ## List of urls to ping
  urls = ["f5.com"]

Then to query Beacon for packet loss percentage in the last hour, use:

POST https://api.cloudservices.f5.com/beacon/v1/metrics
{
  "query": "SELECT mean(\"percent_packet_loss\") AS \"mean_percent_packet_loss\" FROM \"ping\" WHERE time > '2019-10-10T00:00:00Z' GROUP BY time(1h) FILL(none)"
}

Response:

{
  "Results": [
    {
      "Series": [
        {
          "name": "ping",
          "columns": [
            "time",
            "mean_percent_packet_loss"
          ],
          "values": [
            [
              1572134400,
              0
            ],
            [
              1572220800,
              0
            ],
            [
              1572307200,
              0
            ],
            [
              1572393600,
              0
            ]
          ]
        }
      ],
      "Messages": null
    }
  ]
}

Insights

The Insights View page within F5 Beacon is a centralized location to get insightful information about your application ecosystem. Insights are divided into three categories Cost, Operations and Security.

_images/CS-Beacon.Service-Insights.png

Beacon has built-in insights, such as the ‘F5 Assets and Inventory’ insight detailing F5 assets acting as Beacon sources. Built-in insights are currently re-generated every six hours.

Custom Insights

In addition to built-in insights, Beacon users can generate custom insights and publish them to Beacon via the insights API.

See the Beacon GitHub account where there is detailed documentation and sample code for creating custom insights, including a Telegraf Ping plugin example that generates an insight from the metrics reported by the Telegraf Ping plugin.