Cloud Docs Home > F5 BIG-IP Controller for Kubernetes Index

F5 BIG-IP Controller for Kubernetes

Slack

The F5 BIG-IP Controller for Kubernetes lets you manage your F5 BIG-IP device from Kubernetes or OpenShift using either environment’s native CLI/API.

Release Notes

Attributions

Attributions.md

Features

  • Dynamically creates, manages, and destroys BIG-IP objects.
  • Forwards traffic from the BIG-IP device to Kubernetes clusters via NodePorts or ClusterIPs.
  • Support for F5 iApps.
  • Handles F5-specific VirtualServer objects created in Kubernetes.
  • Handles standard Kubernetes Ingress objects using F5-specific extensions.
  • Handles route configuration on the BIG-IP system (OpenShift only).

Guides

See the F5 BIG-IP Controller for Kubernetes user documentation.

Overview

The F5 BIG-IP Controller for Kubernetes is a Docker container that runs in a Kubernetes Pod. It uses F5 Resource s to determine:

  • what objects to configure on your BIG-IP system, and
  • to which Kubernetes Service those objects belong.

The k8s-bigip-ctlr watches the Kubernetes API for the creation, modification, or deletion of Kubernetes objects. For some Kubernetes objects, the Controller responds by creating, modifying, or deleting objects in the BIG-IP system. The k8s-bigip-ctlr handles the following Kubernetes objects:

  • F5 Resource ConfigMap – creates Service-specific frontend virtual servers and/or pools on the BIG-IP system.
  • Kubernetes Ingress – creates a single front-end virtual server on the BIG-IP system that uses L7 policies to route to individual Services.
  • OpenShift Route – enables route-handling (specific to OpenShift).

One Controller can handle a mix of these objects simultaneously. See below for specifics regarding the handling of these objects.

For example, when run in NodePort mode, the k8s-bigip-ctlr does the following:

  1. Discovers a new F5 virtualServer resource.
  2. Creates a new virtual server object in the specified partition on the BIG-IP system. [4]
  3. Creates a pool member on the virtual server for each node in the cluster. [5]
  4. Monitors F5 resources, and linked Kubernetes resources, for changes.
  5. Reconfigures the BIG-IP system when it discovers changes.

The BIG-IP system handles traffic for the Service at the specified virtual address and load balances to all nodes in the cluster. Within the cluster, the allocated NodePort load balances traffic to all pods.

Danger

The k8s-bigip-ctlr monitors the BIG-IP partition it manages for configuration changes. If it discovers changes, the Controller reapplies its own configuration to the BIG-IP system.

F5 does not recommend making configuration changes to objects in any partition managed by the k8s-bigip-ctlr via any other means (for example, the configuration utility, TMOS, or by syncing configuration with another device or service group). Doing so may result in disruption of service or unexpected behavior.

Controller Configuration Parameters

All configuration parameters below are global to the k8s-bigip-ctlr.

General

Parameter Type Required Default Description Allowed Values
log-level string Optional INFO Log level INFO, DEBUG, CRITICAL, WARNING, ERROR
node-poll-interval integer Optional 30 In seconds, interval at which to poll the cluster for its node members.  
python-basedir string Optional /app/python Path to python utilities directory  
verify-interval integer Optional 30 In seconds, interval at which to verify the BIG-IP configuration.  

BIG-IP system

Parameter Type Required Default Description Allowed Values
bigip-partition string Required n/a The BIG-IP partition in which to configure objects.  
bigip-password string Required n/a BIG-IP iControl REST password [6]  
bigip-url string Required n/a BIG-IP admin IP address  
bigip-username string Required n/a BIG-IP iControl REST username [7]  

Kubernetes

Parameter Type Required Default Description Allowed Values
kubeconfig string Optional ./config Path to the kubeconfig file  
namespace string Optional All

Kubernetes namespace(s) to watch

  • may be a comma-separated list
  • watches all namespaces by default
 
namespace-label string Optional n/a Tells the k8s-bigip-ctlr to watch any namespace with this label  
node-label-selector string Optional n/a Tells the k8s-bigip-ctlr to watch only nodes with this label  
pool-member-type string Optional nodeport

The type of BIG-IP pool members you want to create. [7]

Use cluster to create pool members for each of the endpoints for the Service (the pod’s InternalIP)

Use nodeport to create pool members for each schedulable node using the Service’s NodePort.

cluster, nodeport
resolve-ingress-names string Optional n/a

Tells the controller to resolve the first Host in an Ingress resource to an IP address. This IP address will be used as the virtual server address for the Ingress resource.

A value of “LOOKUP” will use local DNS to resolve the Host. Any other value is a custom DNS server and the controller sends resolution queries through that server instead.

Specifying the flag with no argument will default to LOOKUP.

 
running-in-cluster boolean Optional true Indicates whether or not a kubernetes cluster started k8s-bigip-ctlr true, false
use-node-internal boolean Optional true filter Kubernetes InternalIP addresses for pool members true, false
use-secrets boolean Optional true Tells the controller whether or not to load SSL profiles from Kubernetes Secrets for Ingresses and ConfigMaps. If false, the controller will only use profiles from the BIG-IP system. true, false

Note

Use the node-label-selector parameter if you only want the controller to manage specific nodes from the cluster. For example, the BIG-IP device may not be able to reach certain nodes, or the BIG-IP device already manages certain nodes. Therefore, the controller should only watch the nodes that match the environmental constraints (by using a label).

OpenShift SDN

Parameter Type Required Default Description Allowed Values
openshift-sdn-name string Optional n/a

Name of the VXLAN set up on the BIG-IP system that corresponds to an Openshift SDN HostSubnet.

Only applicable in OpenShift.

 

OpenShift Routes

Parameter Type Required Default Description Allowed Values
default-client-ssl string Optional n/a

Specify the name of a user created client ssl profile that will be attached to the route https vserver and used as default for SNI. This profile must have the Default for SNI field enabled.

Only applicable in OpenShift.

 
default-server-ssl string Optional n/a

Specify the name of a user created server ssl profile that will be attached to the route https vserver and used as default for SNI. This profile must have the Default for SNI field enabled.

Only applicable in OpenShift.

 
manage-routes boolean Optional false

Indicates if k8s-bigip-ctlr should handle OpenShift Route objects.

Only applicable in OpenShift.

true, false
route-http-vserver string Optional ose-vserver

The name of the http virtual server for OpenShift Routes.

Only applicable in OpenShift.

 
route-https-vserver string Optional https-ose-vserver

The name of the https virtual server for OpenShift Routes.

Only applicable in OpenShift.

 
route-label string Optional n/a

Tells the k8s-bigip-ctlr to only watch for OpenShift Route objects with the f5type label set to this value.

Only applicable in OpenShift.

 
route-vserver-addr string Optional n/a

Bind address for virtual server for OpenShift Route objects.

Only applicable in OpenShift.

 

Note

If the default-client-ssl or default-server-ssl parameters are not provided, then the controller creates default clientssl and serverssl profiles for the OpenShift Route HTTPS virtual server. The controller sets these profiles as Default for SNI.

F5 Resource ConfigMap Properties

F5 Resource ConfigMap objects tell the k8s-bigip-ctlr how to configure the BIG-IP system. See the Integration Overview for more information about F5 resources.

Property Description Allowed Values
f5type Tells k8s-bigip-ctlr about resources it should watch  
schema Verifies the data blob

f5schemadb://bigip-virtual-server_v0.1.4.json [1]

f5schemadb://bigip-virtual-server_v0.1.3.json

f5schemadb://bigip-virtual-server_v0.1.2.json

data Defines the F5 resource  
frontend Defines object(s) created on the BIG-IP See Frontend
backend Identifes the Kubernets Service acting as the server pool See Backend
[1]Schemas are backwards compatible. F5 recommends using the latest schema in order to make use of the new features available in the controller.

Frontend

virtualServer

The frontend.virtualServer properties define BIG-IP virtual server, pool, and pool member objects.

Property Type Required Default Description Allowed Values
partition string Required   Define the BIG-IP partition to manage  
virtualAddress JSON object Optional   Allocate a virtual address from the BIG-IP  
string Required   Virtual IP address  
  • port
integer Required   Port number  
mode string Optional tcp Set the proxy mode http, tcp
balance string Optional round-robin Set the load balancing mode round-robin
sslProfile [3] JSON object Optional   BIG-IP SSL profile to apply to the virtual server.  
  • f5ProfileName
string Optional  

Name of the BIG-IP SSL profile you want to use.

Uses format partition_name/cert_name

Example: Common/testcert

 
  • f5ProfileNames
array of strings Optional  

Array of BIG-IP SSL profile names.

Each SSL profile name uses the format partition_name/cert_name.

Example:

[
  'Common/testcert1',
  'Common/testcert2'
]
 
[2]The controller supports BIG-IP route domain specific addresses.
[3]If you want to configure multiple SSL profiles, use f5ProfileNames instead of f5ProfileName. The two parameters are mutually exclusive.

If you don’t define bindAddr in the Frontend configuration, you must include it in a Kubernetes Annotation to the ConfigMap. The Controller watches for the annotation key virtual-server.f5.com/ip. This annotation must contain the IP address you want to assign to the virtual server.

  • You can configure an IPAM system to write out an annotation containing the selected IP address.
  • You can check the status.virtual-server.f5.com/ip annotation set by the Controller via the Kubernetes API. This allows you to see the bindAddr assigned to the virtual server.

If you don’t define virtualAddress or bindAddr in the Frontend configuration, the Controller configures and manages pools, pool members, and healthchecks for the Service without a BIG-IP virtual server. In such cases, you should already have a BIG-IP virtual server that handles client connections configured with an iRule or local traffic policy that can forward the request to the correct pool. The stable name of the pool will be the Kubernetes namespace the Service runs in, followed by an underscore, followed by the name of the Service’s ConfigMap. For example: default_myService.

See also

See Manage pools without virtual servers for more information.

iApps

The frontend.virtualServer properties provide the information required to deploy an iApp on the BIG-IP system.

Tip

The iappOptions represent information that the user would provide if deploying the iApp via the BIG-IP configuration utility.

Property Type Required Description Allowed Values
partition string Required The BIG-IP partition you want the k8s-bigip-ctlr to manage.  
iapp string Required BIG-IP iApp template to use to create the application Service. Any iApp template that already exists on the BIG-IP system.
iappPoolMemberTable JSON object Required

Define the name and layout of the pool member table in the iApp.

See iApp Pool Member Table.

 
iappTables JSON object array Optional

Define iApp tables to apply to the Application Service

Example:

"iappTables": {
  "monitor__Monitors":
    {"columns": ["Index", "Name", "Type", "Options"],
     "rows": [[0, "mon1", "tcp", "" ],
              [1, "mon2", "http", ""]]}}"
 
iappOptions key-value object Required Define the App configurations See Controller Configuration Parameters.
iappVariables key-value object Required Define the iApp variables needed for Service creation.  
iApp Pool Member Table

You can use the iappPoolMemberTable option to describe the layout of the pool-member table that the Controller should configure. It is a JSON object with these properties:

  • name (required): A string that specifies the name of the table that contains the pool members.
  • columns (required): An array that specifies the columns that the Controller will configure in the pool-member table, in order.

Each entry in columns is an object that has a name property and either a kind or value property:

  • name (required): A string that specifies the column name.
  • kind: A string that tells the Controller what property from the node to substitute. The Controller supports "IPAddress" and "Port".
  • value: A string that specifies a value. The Controller will not perform any substitution, it uses the value as specified.

For example: If you configure an application with two pods at 1.2.3.4:20123 and 1.2.3.5:20321 and you specify the following JSON:

"iappPoolMemberTable" = {
  "name": "pool__members",
  "columns": [
    {"name": "Port", "kind": "Port"},
    {"name": "IPAddress", "kind": "IPAddress"},
    {"name": "ConnectionLimit", "value": "0"}
  ]
}

the k8s-bigip-ctlr creates the table below on the BIG-IP system.

{
  "name": "pool__members",
  "columnNames": [
    "Port",
    "IPAddress",
    "ConnectionLimit",
  ],
  "rows": [
    {
      "row": [
        "20123",
        "1.2.3.4",
        "0",
      ]
    },
    {
      "row": [
        "20321",
        "1.2.3.5",
        "0",
      ]
    },
  ]
}

You will need to adjust this for the particular iApp template that you are using. One way to discover the format is to configure an iApp manually from a template, then check its configuration using tmsh list sys app Service <appname>.

Backend

The backend section tells the k8s-bigip-ctlr about the Service you want to manage.

Property Type Required Default Description Allowed Values
ServiceName string Required none The Kubernetes Service representing the server pool.  
ServicePort integer Required none Kubernetes Service port number  
healthMonitors JSON object array Optional none Array of TCP or HTTP Health Monitors.  

Ingress Resources

You can use the k8s-bigip-ctlr as a Kubernetes Ingress Controller to expose Services to external traffic.

Supported annotations

Annotation Type Required Description Default
virtual-server.f5.com/ip string Required The IP address you want to assign to the virtual server. N/A
virtual-server.f5.com/partition string Optional The BIG-IP partition in which the Controller should create/update/delete objects for this Ingress. N/A
kubernetes.io/ingress.class string Optional Tells the Controller it should only manage Ingress resources in the f5 class. If defined, the value must be f5. f5
virtual-server.f5.com/balance string Optional Specifies the load balancing mode. round-robin
virtual-server.f5.com/http-port integer Optional Specifies the HTTP port. 80
virtual-server.f5.com/https-port integer Optional Specifies the HTTPS port. 443
virtual-server.f5.com/health JSON object Optional Defines a health monitor for the Ingress resource. N/A
  path string Required [8] The path for the Service specified in the Ingress resource. N/A
  send string Required [8] The send string to set in the health monitor. [9] N/A
  interval integer Required [8] The interval at which to check the health of the virtual server. N/A
  timeout integer Required [8] Number of seconds before the check times out. N/A
ingress.kubernetes.io/allow-http boolean Optional Tells the Controller to allow HTTP traffic for HTTPS Ingress resources. false
ingress.kubernetes.io/ssl-redirect boolean Optional Tells the Controller to redirect HTTP traffic to the HTTPS port for HTTPS Ingress resources (see TLS Ingress resources, below). true

TLS Ingress resources

If the Ingress resource contains a tls section, the allow-http and ssl-redirect annotations provide a method of controlling HTTP traffic. In this case, the Controller uses the value set in the allow-http annotation to enable or disable HTTP traffic. Use the ssl-redirect annotation to redirect all HTTP traffic to the HTTPS Virtual Server.

You can specify one (1) or more SSL profiles in the Ingress resource.

  • Profiles must already exist either in Kubernetes/OpenShift –OR– on the BIG-IP system;
    • If the controller looks for a Kubernetes Secret with the name(s) provided first;
    • if it doesn’t find a matching Secret, the Controller assumes that the name(s) matches a profile that already exists on the BIG-IP system.
    • If naming an existing BIG-IP profile, provide the full path to the profile (for example, /Common/clientssl).

To configure health monitors on your Ingress resource, define the virtual-server.f5.com/health annotation with a JSON object. Provide an array for each path specified in the Ingress resource. For example

{
"path": "ServiceName/path",
"send": "<send string to set in the health monitor>",
"interval": <health check interval>,
"timeout": <number of seconds before the check has timed out>
}

OpenShift Route Resources

Note

You can use OpenShift Route resources in an existing deployment once you replace the OpenShift F5 Router with the BIG-IP Controller.

Supported Route Configurations

Type Client Connection Encrypted Server Connection Encrypted Path Support SSL Termination on BIG-IP Description
Unsecured No No Yes No The BIG-IP system forwards unsecured traffic from the client to the endpoint.
Edge Terminated Yes No Yes Yes

The Controller maintains a new client SSL profile on the BIG-IP system based on the client certificate and key from the Route resource.

  • Set insecureEdgeTerminationPolicy in the Route resource to Allow to enable support for insecure client connections.
  • Set insecureEdgeTerminationPolicy in the Route resource to Redirect to redirect HTTP client connections to the HTTPS endpoint.
Passthrough Terminated Yes Yes No No The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.
Re-encrypt Terminated Yes Yes Yes Yes

The Controller maintains a new BIG-IP client SSL profile based on the client certificate and key from the Route resource.

The Controller maintains a new BIG-IP server SSL profile based on the server CA certificate from the Route resource for re-encrypting the traffic.

The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.

Supported annotations

Annotation Type Required Description Default
virtual-server.f5.com/clientssl string Optional The name of a pre-configured client ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate and key within the Route’s configuration. N/A
virtual-server.f5.com/serverssl string Optional The name of a pre-configured server ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate within the Route’s configuration. N/A
virtual-server.f5.com/health JSON object Optional Defines a health monitor for the Route resource. N/A
  path string Required [8] The path for the Service specified in the Route resource. N/A
  send string Required [8] The send string to set in the health monitor. [9] N/A
  interval integer Required [8] The interval at which to check the health of the virtual server. N/A
  timeout integer Required [8] Number of seconds before the check times out. N/A

Please see the example configuration files for more details.

Example Configuration Files

Footnotes

[4]The k8s-bigip-ctlr creates and manages objects in the BIG-IP partition defined in the F5 resource ConfigMap. It cannot manage objects in the /Common partition.
[5]The k8s-bigip-ctlr forwards traffic to the NodePort assigned to the Service by Kubernetes. See the Kubernetes Services documentation for more information.
[6]You can secure your BIG-IP credentials using a Kubernetes Secret.
[7](1, 2) The BIG-IP user account must have an appropriate role defined. For nodeport type pool members, this role must be either Administrator, Resource Administrator, or Manager. For cluster type pool members, the user account must have either the Administrator or Resource Manager role. See BIG-IP Users for further details.
[8](1, 2, 3, 4, 5, 6, 7, 8) Required if defining the virtual-server.f5.com/health Ingress/Route annotation.
[9](1, 2) See the HTTP monitor settings section of the BIG-IP LTM Monitors Reference Guide for more information about defining send strings.