F5 Container Integrations v1.1

Current Page

Application Services Proxy

Cloud Foundry

Kubernetes

Mesos Marathon

OpenShift

Support

Troubleshooting

Tutorials

Cloud Docs Home > F5 Container Integrations Index

Manage BIG-IP LTM objects in Kubernetes

The BIG-IP Controller for Kubernetes watches the Kubernetes API for Services with associated F5 resources and creates/modifies BIG-IP Local Traffic Manager (LTM) objects accordingly. F5 resources contain the settings BIG-IP Controller should apply to the BIG-IP LTM objects.

Example:

The example virtual server F5 resource JSON blob shown below tells BIG-IP Controller to create one (1) virtual server - with one (1) health monitor and one (1) pool - in the kubernetes partition on the BIG-IP device.

Example F5 Resource definition
// Note: Remove all comments before using
{
  "virtualServer": {
    "backend": {
      "servicePort": 3000,
      "serviceName": "myService",
      "healthMonitors": [{
        "interval": 30,
        "protocol": "http",
        "send": "GET",
        "timeout": 86400
      }]
    },
    "frontend": {
      "virtualAddress": {
        "port": 80,
        // Sets the IP address of the BIG-IP front-end virtual server //
        // omit if you want to create a pool without a virtual server //
        "bindAddr": "173.16.2.2"
      },
      "partition": "kubernetes",
      "balance": "round-robin",
      "mode": "http"
    }
  }
}

Create a BIG-IP front-end virtual server for a Kubernetes Service

Note

The BIG-IP Controller prefaces all BIG-IP virtual server objects with [namespace]_[configmap-name]. For example, default_k8s.vs_173.16.2.2:80, where default is the Kubernetes namespace and k8s.vs is the ConfigMap name.

  1. Create a ConfigMap with the virtual server F5 resource JSON blob in the “data” section.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    # Note:
    # "bindAddr" sets the IP address of the BIG-IP front-end virtual server //
    # omit "bindAddr" if you want to create a pool without a virtual server //
    # (not available in v1.0)
    
    kind: ConfigMap
    apiVersion: v1
    metadata:
      # name of the resource to create on the BIG-IP
      name: k8s.vs
      # The namespace to create the object in.
      # The k8s-bigip-ctlr watches all namespaces by default (as of v1.1).
      # If the k8s-bigip-ctlr is watching a specific namespace(s),
      # this setting must match the namespace of the Service you want to proxy
      # -AND- the namespace(s) the k8s-bigip-ctlr watches.
      namespace: default
      labels: 
        # the type of resource you want to create on the BIG-IP
        f5type: virtual-server
    data:
      # If you're running v1.0, use schema v0.1.2
      schema: "f5schemadb://bigip-virtual-server_v0.1.3.json"
      data: |
        {
          "virtualServer": {
            "backend": {
              "servicePort": 3000,
              "serviceName": "myService",
              "healthMonitors": [{
                "interval": 30,
                "protocol": "http",
                "send": "GET",
                "timeout": 86400
              }]
            },
            "frontend": {
              "virtualAddress": {
                "port": 80,
                "bindAddr": "173.16.2.2"
              },
              "partition": "kubernetes",
              "balance": "round-robin",
              "mode": "http"
            }
          }
        }
    
    
    

    f5-resource-vs-example.configmap.yaml

  2. Upload the ConfigMap to Kubernetes.

    ubuntu@k8s-master:~$ kubectl create -f f5-resource-vs-example.configmap.yaml --namespace=<service-namespace>
    configmap "k8s.vs" created
    
  3. Verify creation of the BIG-IP virtual server.

    TMSH

    admin@(bigip)(cfg-sync Standalone)(Active)(/kubernetes)$ tmsh show ltm virtual
    ------------------------------------------------------------------
    Ltm::Virtual Server: frontend_173.16.2.2_80
    ------------------------------------------------------------------
    Status
      Availability     : available
      State            : enabled
      Reason           : The virtual server is available
      CMP              : enabled
      CMP Mode         : all-cpus
      Destination      : 173.16.2.2:80
    ...
    

Update a BIG-IP virtual server

Use kubectl edit to open the ConfigMap in your default text editor and make your desired changes.

Hint

Kubernetes disregards any breaking or syntactically-incorrect changes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
ubuntu@k8s-master:~$ kubectl edit configmap k8s.vs

# Please edit the object below.
# ...
#
apiVersion: v1
data:
  data: |
    {
      "virtualServer": {
        "backend": {
          "servicePort": 3000,
          "serviceName": "myService",
          "healthMonitors": [{
            "interval": 30,
            "protocol": "http",
            "send": "GET",
            "timeout": 86400
          }]
        },
        "frontend": {
          "virtualAddress": {
            "port": 80,
            "bindAddr": "173.16.2.2"
          },
          "partition": "kubernetes",
          "balance": "round-robin",
          "mode": "http"
        }
      }
    }
  # If you're running v1.0, use schema v0.1.2
  schema: f5schemadb://bigip-virtual-server_v0.1.3.json
kind: ConfigMap
metadata:
  creationTimestamp: 2017-02-14T17:24:34Z
  labels:
    f5type: virtual-server
  name: k8s.vs
  namespace: default

After you save your changes and exit, you can verify the changes using kubectl get.

ubuntu@k8s-master:~$ kubectl get configmap k8s.vs -o yaml

You can also verify the changes on your BIG-IP device using tmsh or the configuration utility.

TMSH

tmsh show ltm virtual

Delete BIG-IP LTM objects

  1. Remove the ConfigMap from the Kubernetes API server to delete the corresponding BIG-IP LTM objects.

    ubuntu@k8s-master:~$ kubectl delete configmap k8s.vs
    configmap "k8s.vs" deleted
    
  2. Verify the BIG-IP LTM objects no longer exist.

    TMSH

    admin@(bigip)(cfg-sync Standalone)(Active)(/kubernetes)$ tmsh show ltm virtual admin@(bigip)(cfg-sync Standalone)(Active)(/kubernetes)$

Create a BIG-IP health monitor for a Kubernetes Service

When running in the default nodeport mode, the BIG-IP Controller for Kubernetes is not aware of Kubernetes node health. Configure BIG-IP LTM health monitors for Kubernetes Services to help ensure that BIG-IP Controller doesn’t send traffic to unhealthy nodes.

  1. Edit the Service definition.

    ubuntu@k8s-master:~$ kubectl edit configmap k8s.vs
    
    # Please edit the object below.
    # ...
    
  2. Define the desired health monitor in the backend section of the F5 resource.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    # Note:
    # "bindAddr" sets the IP address of the BIG-IP front-end virtual server //
    # omit "bindAddr" if you want to create a pool without a virtual server //
    # (not available in v1.0)
    
    kind: ConfigMap
    apiVersion: v1
    metadata:
      # name of the resource to create on the BIG-IP
      name: k8s.vs
      # The namespace to create the object in.
      # The k8s-bigip-ctlr watches all namespaces by default (as of v1.1).
      # If the k8s-bigip-ctlr is watching a specific namespace(s),
      # this setting must match the namespace of the Service you want to proxy
      # -AND- the namespace(s) the k8s-bigip-ctlr watches.
      namespace: default
      labels: 
        # the type of resource you want to create on the BIG-IP
        f5type: virtual-server
    data:
      # If you're running v1.0, use schema v0.1.2
      schema: "f5schemadb://bigip-virtual-server_v0.1.3.json"
      data: |
        {
          "virtualServer": {
            "backend": {
              "servicePort": 3000,
              "serviceName": "myService",
              "healthMonitors": [{
                "interval": 30,
                "protocol": "http",
                "send": "GET",
                "timeout": 86400
              }]
            },
            "frontend": {
              "virtualAddress": {
                "port": 80,
                "bindAddr": "173.16.2.2"
              },
              "partition": "kubernetes",
              "balance": "round-robin",
              "mode": "http"
            }
          }
        }
    
    
    
  3. Use the BIG-IP configuration utility to verify that the health monitor exists.

    Local Traffic ‣ Monitors

Use IPAM to assign IP addresses to BIG-IP LTM virtual servers

The BIG-IP Controller for Kubernetes has a built-in hook that allows you to integrate an IPAM system using a custom plugin. The basic elements required are:

  1. Add an F5 resource ConfigMap set up for unattached pools to the Service definition. The BIG-IP Controller for Kubernetes creates a BIG-IP pool that isn’t attached to a virtual server.

  2. Use your IPAM system to annotate the ConfigMap. The annotation should follow the format virtual-server.f5.com/ip=<ipaddress>.

    The annotation tells the BIG-IP Controller for Kubernetes to create a BIG-IP virtual server with the designated IP address and attach the pool to it.

Connectivity for down or replaced Services

If you need to take down a Kubernetes Service temporarily and want to keep the associated BIG-IP LTM objects, leave the corresponding F5 Resource ConfigMap in place. The BIG-IP Controller will continue to manage the associated BIG-IP LTM objects when the Service comes back up. If you deploy a new Service with the same name as the one you took down, the BIG-IP Controller associates the existing BIG-IP LTM objects with the new Service.

If you take down a Service and want to remove the corresponding BIG-IP LTM objects, delete the F5 Resource ConfigMap.

Manage pools without virtual servers

The BIG-IP Controller for Kubernetes can create and manage BIG-IP Local Traffic Manager (LTM) pools that aren’t attached to a front-end BIG-IP virtual server (also referred to as “unattached pools”). When you create a pool without a virtual server, the BIG-IP Controller for Kubernetes applies the following naming convention to the pool members: <namespace>_<configmap-name>. For example, default_k8s.pool_only.

Important

Your BIG-IP device must have a virtual server with an iRule, or a local traffic policy, that can direct traffic to the unattached pool. After creating an unattached pool, add its member(s) to the iRule or traffic policy to ensure proper handling of client connections to your back-end applications.

Create a pool without a virtual server

  1. Create an F5 resource ConfigMap, omitting the bindAddr field from the virtualAddress section.

    Attention

    As of k8s-bigip-ctlr v1.1.0, you must use F5 schema v0.1.3.

    f5schemadb://bigip-virtual-server_v0.1.3.json

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    kind: ConfigMap
    apiVersion: v1
    metadata:
      # name of the resource to create on the BIG-IP
      name: k8s.pool_only
      # the namespace to create the object in
      # As of v1.1, the k8s-bigip-ctlr watches all namespaces by default
      # If the k8s-bigip-ctlr is watching a specific namespace(s),
      # this setting must match the namespace of the Service you want to proxy
      # -AND- the namespace(s) the k8s-bigip-ctlr watches
      namespace: default
      labels:
        # the type of resource you want to create on the BIG-IP
        f5type: virtual-server
    data:
      schema: "f5schemadb://bigip-virtual-server_v0.1.3.json"
      data: |
        {
          "virtualServer": {
            "backend": {
              "servicePort": 3000,
              "serviceName": "myService",
              "healthMonitors": [{
                "interval": 30,
                "protocol": "http",
                "send": "GET",
                "timeout": 86400
              }]
            },
            "frontend": {
              "virtualAddress": {
                "port": 80
              },
              "partition": "kubernetes",
              "balance": "round-robin",
              "mode": "http"
            }
          }
        }
    

    f5-resource-pool-only-example.configmap.yaml

  2. Upload the ConfigMap to Kubernetes.

    ubuntu@k8s-master:~$ kubectl create -f f5-resource-pool-only-example.configmap.yaml --namespace=<service-namespace>
    configmap "k8s.pool_only" created
    

Attach a pool to a virtual server

  1. Add the desired bindAddr (in other words, the virtual server IP address) to the F5 Resource ConfigMap using kubectl edit.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    ubuntu@k8s-master:~$ kubectl edit configmap k8s.pool_only
    
    # Please edit the object below.
    # ...
    #
    apiVersion: v1
    data:
      data: |
        {
          "virtualServer": {
            "backend": {
              "servicePort": 3000,
              "serviceName": "myService",
              "healthMonitors": [{
                "interval": 30,
                "protocol": "http",
                "send": "GET",
                "timeout": 86400
              }]
            },
            "frontend": {
              "virtualAddress": {
                "port": 80,
                "bindAddr": "1.2.3.4" \\ add this line, using a valid IP address
              },
              "partition": "kubernetes",
              "balance": "round-robin",
              "mode": "http"
            }
          }
        }
      schema: f5schemadb://bigip-virtual-server_v0.1.3.json
    kind: ConfigMap
    metadata:
      creationTimestamp: 2017-02-14T17:24:34Z
      labels:
        f5type: virtual-server
      name: k8s.pool_only
      namespace: default
    
  2. Verify the changes using kubectl get.

    ubuntu@k8s-master:~$ kubectl get configmap k8s.pool_only -o yaml
    
  3. Use the BIG-IP configuration utility to verify the pool attached to the virtual server.

    Local Traffic ‣ Virtual Servers

Tip

You can use an IPAM system to populate the bindAddr field automatically.

Delete an “unattached” pool

  1. Remove the ConfigMap from the Kubernetes API server.

    ubuntu@k8s-master:~$ kubectl delete configmap k8.pool_only
    configmap "k8s.pool_only" deleted
    
  2. Use the BIG-IP configuration utility to verify deletion of the pool.

    Local Traffic ‣ Pools

Detach a pool from a virtual server

If you want to delete a front-end BIG-IP virtual server, but keep its associated pool(s)/pool member(s):

  1. Remove the bindAddr field from the virtual server F5 resource ConfigMap.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    ubuntu@k8s-master:~$ kubectl edit configmap k8s.vs
    
    # Please edit the object below.
    # ...
    #
    apiVersion: v1
    data:
      data: |
        {
          "virtualServer": {
            "backend": {
              "servicePort": 3000,
              "serviceName": "myService",
              "healthMonitors": [{
                "interval": 30,
                "protocol": "http",
                "send": "GET",
                "timeout": 86400
              }]
            },
            "frontend": {
              "virtualAddress": {
                "port": 80,
                "bindAddr": "1.2.3.4" \\ remove this line
              },
              "partition": "kubernetes",
              "balance": "round-robin",
              "mode": "http"
            }
          }
        }
      schema: f5schemadb://bigip-virtual-server_v0.1.3.json
    kind: ConfigMap
    metadata:
      creationTimestamp: 2017-02-14T17:24:34Z
      labels:
        f5type: virtual-server
      name: k8s.vs
      namespace: default
    
  2. Verify the changes using kubectl get.

    ubuntu@k8s-master:~$ kubectl get configmap k8s.vs -o yaml
    
  3. Use the BIG-IP configuration utility to verify the virtual server no longer exists.

    Local Traffic ‣ Virtual Servers