Work with F5 Beacon

F5 Beacon tracks the health and status of different application components, defines application structure and allows users to create customized dashboards to get a global view of their applications.

The main entities in Beacon are:

  • Source - sends analytics data and stats on components; for example, BIG-IP sources send stats on virtual servers.
  • Component - contains state for a single application component, wheather it is the health state or other metrics
  • Application - Beacon’s representation of an application, depicted as a tree structure composed of application nodes
  • Application node - a single entity in the application tree structure with health derived from one or more metrics and/or dependencies on other nodes
  • Widget - a visualization module that utilizes Beacon data to provide insight on the application portfolio
  • Dashboard - a customizable grouping of widgets

To utilize Beacon, users need to configure sources to publish analytic data to Beacon, onboard applications, setup setup applications’ health source settings, and create a dashboard with widgets to gain insight on application status.

In this tutorial, we will demonstrate how to

  1. Configure a BIG-IP source to publish analytics data to Beacon
  2. Review component health data published by the source
  3. Onboard an application and define health setting
  4. Create a dashboard which visualizes the application portfolio

Use the F5 Beacon portal

Access the F5 Beacon portal by using the Beacon tab in the Cloud Services portal navigation menu.

_images/CS-Beacon.Service.png

Beacon has six menu items:

  • Dashboard View - View the configured dashboards
  • Insights View - View both built-in and custom insights by category and severity
  • Application Inventory - View the list of applications and their health status
  • Component Inventory - View the list of components and their health status
  • Dashboard Management - Create, modify and delete dashboards
  • Source Configuration - Configure sources to provide application analytics data

Configure source

A source is any entity that sends component data to Beacon. Currently, the main source of information is a BIG-IP device. In order to configure BIG-IP to send analytics data to Beacon, we must configure the device to utilize telemetry streaming, part of the F5 Automation Toolchain. To add a BIG-IP device as a source to Beacon, ensure that telemetry streaming is installed on that device.

Add a BIG-IP device source

To add a BIG-IP device source:

  1. Within the portal, click the Beacon tab in the side menu.

  2. Via the drop down menu, select Source Configuration.

    _images/CS-Beacon.Service-Source-Config.png
  3. Navigate to the Integrations tab and click the link to view the instructions for configuring a BIG-IP integration.

    _images/CS-Beacon.Service-Source-Config-Create-Integration.png
  4. The instructions provide a link to create a token for use on BIG-IP. On the Tokens tab, click Create.

    _images/CS-Beacon.Service-Source-Config-Create-Token.png
  5. The Create Token side panel will appear. Provide a name and optional description and click Create Token.

    _images/CS-Beacon.Service-Source-Config-Token-Table.png
  6. Select the token in the grid to see its details. Copy the access token for use in the Telemetry Streaming declaration.

    _images/CS-Beacon.Service-Source-Config-Token-Instructions.png
  7. After following the instructions for configuring Telemetry Streaming, navigate to the Sources tab and see the newly added BIG-IP source.

Note

  1. It can take up to a minute for a device to be processed and appear in the list of sources.
  2. If the device still does not appear, validate the device has outbound connectivity to Beacon and/or check the Telemetry Streaming log.

View Component Inventory

Once a source is added, we can examine associated components in the Component Inventory.

  1. Within the portal, click the Beacon tab in the side menu.

  2. Via the drop down menu, select Component Inventory. The inventory includes a list of all components with their current health and associated source.

    _images/CS-Beacon.Service-Component-inventory.png

On-board an application

Applications in Beacon are composed of one or more nodes. For example, an application may be composed of an API service and data processing services. Nodes can be nested in a tree structure to represent the dependencies that they have on each other.

_images/CS-Beacon.Service-App-map.png

The health for each node is derived from a combination of health sources and dependencies on other nodes. The components visible in the component inventory can be used as health sources for nodes. The mapping of components with health information to nodes and the relationships between nodes provide a live picture of holistic application health.

_images/CS-Beacon.Service-App-map-health.png

To create an application, utilize the Beacon Create Application API.

Note

Download the F5 Cloud Services Beacon Postman collection: Beacon Collection

Authenticate against F5 Cloud Service API

To send an API request to any F5 Cloud Service API, a bearer token must be obtained, see API overviews and example for additional detail.

Specify preferred account header in a multiple accounts/divisions scenario

A user can be associated with multiple accounts or have configured divisions. In these scenarios, to disambiguate the account information, a header named X-F5aaS-Preferred-Account-Id needs to be added with the account ID value.

Account membership information is available via the following APIs.

Retrieve the user ID and primary accountID via:

GET https://api.cloudservices.f5.com/v1/svc-account/user
{
    "id": "u-aaPgo44WXX",
    "email": "user@f5.com",
    "first_name": "First name",
    "last_name": "Last name",
    "phone": "+1 (111) 111-1111",
    "primary_account_id": "a-aaT6XYZa9j",
    "status": "active",
    "email_confirmed": true,
    "phone_confirmed": false,
    "unconfirmed_email": "",
    "time_zone": "",
    "preferred_language": "",
    "user_email_history": [],
    "current_password": "",
    "create_time": "2019-04-30T19:08:47.759310Z",
    "update_time": "2019-04-30T19:08:47.759310Z",
    "activate_time": null,
    "delete_time": null,
    "reset_password_sent_time": null,
    "reset_password_time": null,
    "email_confirmation_sent_time": "2019-05-28T21:05:45.927619Z",
    "email_confirmation_time": "2019-05-28T21:05:45.927619Z",
    "phone_confirmation_sent_time": null,
    "phone_confirmation_time": null
}

Retrieve account memberships via:

GET https://api.cloudservices.f5.com/v1/svc-account/users/<USER_ID>/memberships
{
    "memberships": [
        {
            "account_id": "a-aaSXXdAYYY2",
            "user_id": "u-aaPgo44WXX",
            "role_id": "r-6Ie62X-mR",
            "user": {
                "id": "uu-aaPgo44WXX",
                "email": "user@f5.com",
                "first_name": "First name",
                "last_name": "Last name",
                "phone": "+1 (111) 111-1111",
                "create_time": "2019-04-30T19:08:47.759310Z",
                "update_time": "2019-04-30T19:08:47.759310Z"
            },
            "account_name": "NW - Zone",
            "role_name": "privileged-user",
            "level": "1",
            "signup_provider": "standard",
            "create_time": "2019-08-13T11:20:21.687740Z",
            "update_time": "2019-08-13T11:20:21.687740Z"
        },
        {
            "account_id": "a-aahXYZLa7H",
            "user_id": "u-aaPgo44WXX",
            "role_id": "r-6Ie62X-mR",
            "user": {
                "id": "u-aaPgo44WXX",
                "email": "user@f5.com",
                "first_name": "First name",
                "last_name": "Last name",
                "phone": "+1 (111) 111-1111",
                "create_time": "2019-04-30T19:08:47.759310Z",
                "update_time": "2019-04-30T19:08:47.759310Z"
            },
            "account_name": "f5",
            "role_name": "privileged-user",
            "level": "0",
            "signup_provider": "standard",
            "create_time": "2019-05-28T21:05:45.897722Z",
            "update_time": "2019-05-28T21:05:45.897722Z"
        },
        {
            "account_id": "a-ahhhZMXXJU",
            "user_id": "u-aaPgo44WXX",
            "role_id": "r-LIee2X-iR",
            "user": {
                "id": "u-aaPgo44WXX",
                "email": "user@f5.com",
                "first_name": "First name",
                "last_name": "Last name",
                "phone": "+1 (111) 111-1111",
                "create_time": "2019-04-30T19:08:47.759310Z",
                "update_time": "2019-04-30T19:08:47.759310Z"
            },
            "account_name": "HR",
            "role_name": "owner",
            "level": "1",
            "signup_provider": "standard",
            "create_time": "2019-05-28T21:01:39.001499Z",
            "update_time": "2019-05-28T21:01:39.001499Z"
        },
        {
            "account_id": "a-aaT6XYZa9j",
            "user_id": "u-aaPgo44WXX",
            "role_id": "r-LIee2X-iR",
            "user": {
                "id": "u-aaPgo44WXX",
                "email": "user@f5.com",
                "first_name": "First name",
                "last_name": "Last name",
                "phone": "+1 (111) 111-1111",
                "create_time": "2019-04-30T19:08:47.759310Z",
                "update_time": "2019-04-30T19:08:47.759310Z"
            },
            "account_name": "F5 Networks",
            "role_name": "owner",
            "level": "0",
            "signup_provider": "standard",
            "create_time": "2019-04-30T19:08:47.814373Z",
            "update_time": "2019-04-30T19:08:47.814373Z"
        }
    ]
}

Note

It is recommended to always set the preferred account header to prevent future issues.

Create application API reference

The Beacon Create Application API creates an application, which consists of:

  • The tree structure, starting with the application at the top. Below the application are dependency nodes, which themselves may have dependencies. For any node in the tree, health sources can be defined.
  • Each node in the tree structure has the following fields:
  • name - Display name
  • description - Node description
  • labels - Arbitrary key/value labels can be added to provide metadata or groupings for your applications.
  • healthSourceSettings - These represent the metric used to determine node health of an application. When the app is displayed, each health source is resolved to a concrete health value and can be continually updated. A health source includes:
    • metrics - A list of to use to determine the node health.
  • dependencies - An array of dependencies, each of which is another node and nested dependencies.

Retrieve the list of components via:

POST -d <see below> https://api.cloudservices.f5.com/beacon/v1/applications
{
  "name": "App",
  "description": "An example for an Beacon application",
  "labels": {
    "region": "New York",
    "owner": "ABC team"
  },
  "dependencies": [{
    "name": "API service",
    "dependencies": [{
      "name": "API Gateway",
      "healthSourceSettings": {
        "metrics": [
          {
            "measurementName": "BeaconHealth",
            "tags": {
              "name": "/Common/apiGateWayVirtualServer",
              "source": "my-big-ip1.example.com"
            }
          }
        ]
      }
    },
      {
        "name": "Backend API",
        "dependencies": [{
          "name": "Database service",
          "healthSourceSettings": {
            "metrics": [
              {
                "measurementName": "BeaconHealth",
                "tags": {
                  "name": "/Common/databaseVirtualServer",
                  "source": "my-big-ip2.example.com"
                }
              }
            ]
          }
        }]
      }
    ]
  },
    {
      "name": "Data Process",
      "dependencies": [{
        "name": "Spark cluster"
      },
        {
          "name": "Data Process API",
          "healthSourceSettings": {
            "metrics": [
              {
                "measurementName": "BeaconHealth",
                "tags": {
                  "name": "/Common/sparkVirtualServer1",
                  "source": "my-big-ip3.example.com"
                }
              },
              {
                "measurementName": "BeaconHealth",
                "tags": {
                  "name": "/Common/sparkVirtualServer2",
                  "source": "my-big-ip3.example.com"
                }
              }
            ]
          }
        }
      ]
    }
  ]
}

The Create Application API returns the same body provided as input, but with an additional application “id” field.

Dashboards

Dashboards are used to visualize the current and past state of the application portfolio. A set of widgets is provided, each with specific visualization, such as application health history. Widgets are added to dashboards to create customized views. The Dashboard Management page is used to create, modify and delete dashboards. The Dashboard View page is used to display dashboards.

Add a dashboard

To create a new dashboard:

  • Via the Beacon drop down menu, select Dashboard Management. A list of existing dashboards is displayed. To create a new dashboard, click Create.

    _images/CS-Beacon.Service-Dashboard-mgmt.png
  • On the dashboard creation page, there are two tabs, Properties and Layout.

    • On the Properties tab, provide a name and optional description.

      _images/CS-Beacon.Service-Dashboard-mgmt-layout.png
    • Navigate to the Layout tab and construct the widget layout by dragging widgets onto the dashboard. There are five types of widgets available:

      • Treemap - Display application counts grouped by current health and a specified custom label.
      • Application Health History - Display application counts grouped by health over time.
      • App map - Display the node layout of a single specified application.
      • Events - Display details about application health transitions and trends over time.
      • Insights - Display counts of insights grouped by category and severity.
    • When complete, click Save Dashboard.

View dashboards

To view existing dashboards:

  1. Via the Beacon drop down menu, select Dashboard View. Each existing dashboard is available as a separate tab. In addition to showing the widgets configured, each dashboard includes a list of Application Details. The applications included in the list change based on any filtering applied. Clicking on some widgets automatically applies filtering to the list.

    _images/CS-Beacon.Service-Dashboard-view.png

Edit dashboards

To edit an existing dashboard:

  • Via the Beacon drop down menu, select Dashboard Management. Click on the dashboard name.
  • Editing a dashboard is similar to what was described in the Add a dashboard section.

Delete dashboards

To delete an existing dashboard:

  • Via the Beacon drop down menu, select Dashboard Management.
  • Select the checkbox for the dashboard and click the delete button at the top right of the grid. A popup will appear to confirm the deletion.

Declare API

By using the declare API, you can add and modify, or remove multiple applications with a single call. The declare API path is https://api.cloudservices.f5.com/beacon/v1/declare, where you can POST a declaration of your applications.

Declare API request:

{
    "action": "deploy",
    "declaration": [{
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "AccountingApp",
                "description": "An accounting Application",
                "labels": {
                    "department": "Accounting",
                    "location": "Azure"
                }
            }
        }
    ]
}

Declare has two fields:

  • action - The type of operation to apply to the body. Valid values are “deploy”, “dry-run”, and “remove”. If not provided, the default is “deploy”.
  • declaration - An array field containing configuration resources.

Each declaration resource contains metadata with the following:

  • version - The API version to use to create the resource.

Now that we understand the schema of the declare API, let us see how we can utilize it to manage our applications.

Deploy Action - Creating resources

Use the following POST to create Beacon applications. The deploy action is asynchronous – the API stores the task and processes it in the background. The reply will contain a task ID, allowing you to track the progress of the task.

Deploy action - Create new resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "deploy",
    "declaration": [{
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "AccountingApp",
                "description": "An accounting Application",
                "labels": {
                    "department": "Accounting",
                    "location": "Azure"
                },
                "dependencies": [{
                    "name": "AccountingDB"
                }]
            }
        },
        {
            "metadata": {
                "version": "v2"
            },
            "application": {
                "name": "FinanceApp",
                "description": "An finance Application",
                "labels": {
                    "department": "Finance",
                    "location": "AWS"
                },
                "dependencies": [{
                    "name": "FinanceDB"
                }]
            }
        }
    ]
}

A deploy command with a declaration of an existing resource will update that resource to the state defined in the declaration input. The example below will update the “AccountingApp” with a new location label. It will not modify any other objects beside “AccountingApp”.

Deploy action - Update existing resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "deploy",
    "declaration": [{
        "metadata": {
            "version": "v1"
        },
        "application": {
            "name": "AccountingApp",
            "description": "An accounting Application",
            "labels": {
                "department": "Accounting",
                "location": "AWS" <<<< updated field
            },
            "dependencies": [{
                "name": "AccountingDB"
            }]
        }
    }]
}

Remove action - Delete resources

The “remove” action with the POST body below explicitly deletes both “AccountingApp” and “FinanceApp”, other resources will not be impacted.

Remove action - Delete resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "remove",
    "declaration": [{
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "AccountingApp",
                "description": "An accounting Application",
                "labels": {
                    "department": "Accounting",
                    "location": "AWS"
                },
                "dependencies": [{
                    "name": "AccountingDB"
                }]
            }
        },
        {
            "metadata": {
                "version": "v1"
            },
            "applications": {
                "name": "FinanceApp",
                "description": "A Finance Application",
                "labels": {
                    "department": "Finance",
                    "location": "Azure"
                },
                "dependencies": [{
                    "name": "MarketingDB"
                }]
            }
        }
    ]
}

Alternatively, you can also send just the resource identification keys

Remove action - Delete resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "remove",
    "declaration": [{
            "application": {
                "name": "AccountingApp"
            }
        },
        {
            "application": {
                "name": "FinanceApp"
            }
        }
    ]
}

Dry-Run action - Validate resources

The “dry-run” action will validate the supplied resources and notify what operation will occur if a subsequent deploy action occurs.

Dry-run action - Validate resources:

POST https://api.cloudservices.f5.com/beacon/v1/declare
{
    "action": "dry-run",
    "declaration": [{
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "AccountingApp"
                ...
            }
        },
        {
            "metadata": {
                "version": "v1"
            },
            "application": {
                "name": "MarketingApp"
                ...
            }
        }
    ]
}

The reply metadata contains an operation field indicating if it is an update ‘operation’ when the items already exist or create ‘operation’ when it is a new item.

Response:

Declare API "dry-run" reply
{
    "action": "dry-run",
    "declaration": [{
            "metadata": {
                "version": "v1",
                "operation": "update" <<<<
            },
            "application": {
                "name": "MonitorAccounting"
            }
        },
        {
            "metadata": {
                "version": "v1",
                "operation": "add" <<<<
            },
            "application": {
                "name": "MonitorMarketing"
            }
        }
    ]
}

Metrics

Beacon provides the ability to store metrics and subsequently query them to generate insights and publish them to dashboards. To send metrics to Beacon, you can utilize Telegraf, the open-source server agent that can collect metrics from multiple sources and systems.

Setting up Telegraf with Beacon

Telegraf has a wide range of input plugins that can gather metrics and send them to Beacon. For a detailed list, see Telegraf plugins. To get started with Telegraf, see Getting started with Telegraf and follow the setup instructions.

Select an Input plugin based on the type of metrics you want to collect. Use the HTTP plugin for the Output plugin.

# A plugin that can transmit metrics over HTTP
[[outputs.http]]

  ## URL is the address to send metrics to
  url = "https://ingestion.ovr.prd.f5aas.com:50443/beacon/v1/ingest-metrics"

  ## Timeout for HTTP message
  timeout = "120s"

  ## HTTP method
  method = "POST"

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  insecure_skip_verify = true

  ## Data format to output.
  data_format = "influx"

  ## HTTP Content-Encoding for write request body
  content_encoding = "identity"

  ## Additional HTTP headers
  [outputs.http.headers]
    Content-Type = "text/plain; charset=utf-8"
    X-F5-Ingestion-Token = "...."
  • url - The URL field points to Beacon ingestion endpoint
  • tls - The TLS configuration
  • outputs.http.headers - Add a X-F5-Ingestion-Token header containing an Beacon source token, which is used for authentication

Once Telegraf is configured to run with the outputs.http plugin, Beacon will start receiving metrics sent by the Telegraf agent.

Querying Beacon metrics

You can use the metrics API to query Beacon for metrics. Beacon is using InfluxDB to store metric data and as a result, the query language is InfluxQL. See https://docs.influxdata.com/influxdb/v1.7/query_language/spec/ for syntax details on how to execute a metric query.

For example, if using the Ping input plugin to publish ping results:

## Ping given url(s) and return statistics
[[inputs.ping]]
  ## List of urls to ping
  urls = ["f5.com"]

Then to query Beacon for packet loss percentage in the last hour, use:

POST https://api.cloudservices.f5.com/beacon/v1/metrics
{
  "query": "SELECT mean(\"percent_packet_loss\") AS \"mean_percent_packet_loss\" FROM \"ping\" WHERE time > '2019-10-10T00:00:00Z' GROUP BY time(1h) FILL(none)"
}

Response:

{
  "Results": [
    {
      "Series": [
        {
          "name": "ping",
          "columns": [
            "time",
            "mean_percent_packet_loss"
          ],
          "values": [
            [
              1572134400,
              0
            ],
            [
              1572220800,
              0
            ],
            [
              1572307200,
              0
            ],
            [
              1572393600,
              0
            ]
          ]
        }
      ],
      "Messages": null
    }
  ]
}

Insights

The Insights View page within F5 Beacon is a centralized location to get insightful information about your application ecosystem. Insights are divided into three categories Cost, Operations and Security.

_images/CS-Beacon.Service-Insights.png

Beacon has built-in insights, such as the ‘F5 Assets and Inventory’ insight detailing F5 assets acting as Beacon sources. Built-in insights are currently re-generated every six hours.