F5 Container Integrations

Current Page

Cloud Foundry

Kubernetes / OpenShift

Mesos Marathon

Support

Troubleshooting

Tutorials


View related articles on DevCentral

Cloud Docs Home > F5 Container Integrations Index

Use Ingress Resources to Expose Kubernetes Services to External Traffic

You can use the BIG-IP Controller for Kubernetes as an Ingress Controller in Kubernetes.

Tasks
Step Description
Create a BIG-IP Self IP address for the virtual server.
Define Virtual Server Ingress Annotations in an Ingress Resource.
Add a BIG-IP Health Monitor to the virtual server for an Ingress.
Upload the Ingress to the API server.
Verify object creation on the BIG-IP system.

Overview

The BIG-IP Controller uses the Kubernetes Ingress resource to expose Services to external traffic as follows:

  • creates a BIG-IP virtual server for the Ingress Resource,
  • creates a pool on the virtual server for each Service in the Ingress’ path.

The BIG-IP Controller supports the following Ingress resource types:

The BIG-IP Controller has a set of supported Ingress annotations that let you define the objects you want to create on the BIG-IP system.

Attention

The BIG-IP Controller creates one BIG-IP virtual server for each Ingress resource. If the Ingress resource incorporates multiple Services, the BIG-IP Controller creates one pool for each Service.

If you set allowHttp or sslRedirect to “True”, the Controller creates two virtual servers – one for HTTP and one for HTTPS.

IP address assignment

The BIG-IP Controller supports the following options for IP address assignment. See the k8s-bigip-ctlr configuration parameters table for more information about these settings.

Default IP address

You can set a default IP address for Ingress resources. To do so:

  1. Add the --default-ingress-ip config parameter to the k8s-bigip-ctlr Deployment.
  2. Add the annotation virtual-server.f5.com/ip="controller-default" to your Ingress resource.

The BIG-IP Controller will replace “controller-default” with the IP address provided for the default-ingress-ip parameter.

When you define a default ingress IP address, each Ingress resource configured to use the “controller-default” IP shares the same BIG-IP virtual server. The BIG-IP Controller attaches a separate policy to the virtual server for each Ingress to ensure correct traffic routing for those resources.

Important

You can only define one --default-ingress-ip per BIG-IP Controller instance.

If you’re using multiple Controllers to monitor separate namespaces, you can define a default IP address for each Controller. This type of deployment allows you to isolate the VIPs in each namespace from each other.

DNS lookup

The BIG-IP Controller uses DNS lookup to resolve hostnames by default (as of v1.3.0). The BIG-IP Controller attempts to resolve the first hostname provided in the spec.rules.host section of the Ingress Resource. It then assigns the resolved host’s IP address to the Ingress’ virtual server.

Unattached pools

You can create unattached pools for the Services defined in the Ingress resource. To do so, just omit the virtual-server.f5.com/ip= annotation from your Ingress resource.

You can then use an IPAM system to assign an IP address and attach a virtual server to the pools, or use another means to manually route traffic to the pools on the BIG-IP system.

Deployments using multiple Ingress Controllers

Note

In Kubernetes, the Ingress resource’s ingress.class property is unset by default. The BIG-IP Controller automatically manages all Ingress resources that don’t have an ingress.class` defined.

If you’re using more than one Ingress Controller to manage your Ingress resources:

  • Set the ingress.class property to “f5” for all Ingress resources you want the BIG-IP Controller to manage.

    kubernetes.io/ingress.class="f5"
    
  • Define the ingress.class as appropriate for the Ingress resources managed by other Ingress Controllers. The BIG-IP Controller ignores Ingress resources that have any ingress.class other than “f5”.

Initial Setup

Allocate a Self IP address from the external network on the BIG-IP system. You’ll assign this IP address to the Ingress resource’s BIG-IP virtual server.

Note

  • If you intend to create unattached pools (pools without a virtual server), you will need to set up another way to route traffic to the pools on the BIG-IP system.
  • If you have already assigned a default IP address to the Controller, you may skip this step.

Set Virtual Server Ingress Annotations using kubectl

Add the supported Ingress annotations to any existing Ingress resource using kubectl annotate. The examples below demonstrate correct usage on the command line.

  • Assign an IP address to the Ingress:

    kubectl annotate ingress myIngress virtual-server.f5.com/ip="1.2.3.4"
    
  • Use the default IP address assigned in the k8s-bigip-ctlr Deployment:

    kubectl annotate ingress myIngress virtual-server.f5.com/ip="controller-default"
    
  • Set the desired port:

    kubectl annotate ingress myIngress virtual-server.f5.com/http-port="80"
    
  • Set the BIG-IP partition:

    kubectl annotate ingress myIngress virtual-server.f5.com/partition="k8s"
    
  • Set the load balancing method:

    kubectl annotate ingress myIngress virtual-server.f5.com/balance="round-robin"
    
  • Define a BIG-IP health monitor:

    kubectl annotate ingress myIngress virtual-server.f5.com/health='[{"path": "svc1.example.com/app1", "send": "HTTP GET /health/svc1", "interval": 5, "timeout": 10}]'
    
  • Redirect HTTP requests to HTTPS:

    kubectl annotate ingress myIngress ingress.kubernetes.io/ssl-redirect="true"
    
  • Deny HTTP requests:

    kubectl annotate ingress myIngress ingress.kubernetes.io/allow-http="false"
    
  • Assign the F5 Ingress class to avoid conflicts with other Ingress Controllers:

    kubectl annotate ingress myIngress kubernetes.io/ingress.class="f5"
    

Define Virtual Server Ingress Annotations in an Ingress Resource

When creating a new Ingress Resource, include the supported Ingress annotations as needed. The annotations must be valid JSON.

Add a BIG-IP Health Monitor to the virtual server for an Ingress

Add the virtual-server.f5.com/health annotation to your Ingress resource. The example below shows the correct usage.

Health Monitor Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ing1
  namespace: default
  annotations:
    # Provide an IP address for the BIG-IP virtual server;
    # Set to "controller-default" if you want to use the Controller's
    # "default-ingress-ip" (introduced in v1.4.0)
    virtual-server.f5.com/ip: "controller-default"
    virtual-server.f5.com/partition: "k8s"
    virtual-server.f5.com/health: |
      [
        {
          "path":     "svc1.example.com/app1",
          "send":     "HTTP GET /health/app1",
          "interval": 5,
          "timeout":  10
        }, {
          "path":     "svc2.example.com/app2",
          "send":     "HTTP GET /health/app2",
          "interval": 5,
          "timeout":  5
        }
      ]
spec:
  rules:
  - host: svc1.example.com
    http:
      paths:
      - backend:
          serviceName: svc1
          servicePort: 8080
        path: /app1
  - host: svc2.example.com
    http:
      paths:
      - backend:
          serviceName: svc2
          servicePort: 9090
        path: /app2

Secure an Ingress using TLS

You can secure an Ingress using Secrets or BIG-IP SSL profiles.

  1. Specify the SSL profile(s) or the Secret containing the cert and key in the spec.tls section of the Ingress resource.
  2. Add the ingress.kubernetes.io/ssl-redirect annotation (OPTIONAL; defaults to "true").
  3. Add the ingress.kubernetes.io/allow-http annotation (OPTIONAL; defaults to "false").

Note

  • You can specify one or more SSL profiles in your Ingress resource.
  • If you specify a spec.tls section without providing the TLS Ingress properties, the BIG-IP device uses its local traffic policies to redirect HTTP requests to HTTPS.

See also

Refer to the Kubernetes TLS Ingress documentation for details regarding supported port(s) and termination.

BIG-IP SSL profiles

TLS Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingressTLS
  namespace: default
  annotations:
    # Provide an IP address for the BIG-IP virtual server;
    # Set to "controller-default" if you want to use the Controller's
    # "default-ingress-ip" (introduced in v1.4.0)
    virtual-server.f5.com/ip: "1.2.3.4"
    # Specify the BIG-IP partition where the Controller should create the virtual server.
    virtual-server.f5.com/partition: "k8s"
    # Allow/deny TLS connections
    ingress.kubernetes.io/ssl-redirect: "true"
    # Allow/deny HTTP connections
    ingress.kubernetes.io/allow-http: "false"
spec:
  tls:
    # Provide the BIG-IP SSL Profile you want to use.
    # Follows the format "/partition/profile_name".
    - secretName: /Common/clientssl
  backend:
    # Provide the name of a single Kubernetes Service you want to expose to external
    # traffic using TLS
    serviceName: myService
    servicePort: 443

f5-k8s-ingress-tls.yaml

Kubernetes Secrets

TLS Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingressTLS
  namespace: default
  annotations:
    # Provide an IP address for the BIG-IP virtual server;
    # Set to "controller-default" if you want to use the Controller's
    # "default-ingress-ip" (introduced in v1.4.0)
    virtual-server.f5.com/ip: "1.2.3.4"
    # Specify the BIG-IP partition where the Controller should create the virtual server.
    virtual-server.f5.com/partition: "k8s"
    # Allow/deny TLS connections
    ingress.kubernetes.io/ssl-redirect: "true"
    # Allow/deny HTTP connections
    ingress.kubernetes.io/allow-http: "false"
spec:
  tls:
    # Provide the name of the Secret containing the cert and key you want to use.
    - secretName: myTLSSecret
  backend:
    # Provide the name of a single Kubernetes Service you want to expose to external
    # traffic using TLS
    serviceName: myService
    servicePort: 443


apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: no-rules-map
spec:
  tls:
  - secretName: testsecret
  backend:
    serviceName: s1
    servicePort: 80

f5-k8s-ingress-tls-secret.yaml

See Example Ingress Resources

Single Service

A Single Service Ingress creates a BIG-IP virtual server and server pool for a single Kubernetes Service.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: singleIngress1
  namespace: default
  annotations:
    # Provide an IP address for the BIG-IP virtual server;
    # Set to "controller-default" if you want to use the Controller's
    # "default-ingress-ip" (introduced in v1.4.0)
    virtual-server.f5.com/ip: "1.2.3.4"
    # Specify the BIG-IP partition where the Controller should create the virtual server.
    virtual-server.f5.com/partition: "k8s"
spec:
  backend:
    # The name of the Service you want to expose to external traffic
    serviceName: myService
    servicePort: 80

f5-k8s-single-ingress.yaml

Simple Fanout

A Simple Fanout Ingress creates a BIG-IP virtual server and pools for a group of Kubernetes Services (one pool per Service).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ing-fanout
  namespace: default
  annotations:
    # Provide an IP address for the BIG-IP virtual server;
    # Set to "controller-default" if you want to use the Controller's
    # "default-ingress-ip" (introduced in v1.4.0)
    virtual-server.f5.com/ip: "1.2.3.4"
    # Specify the BIG-IP partition where the Controller should create the virtual server.
    virtual-server.f5.com/partition: "k8s"
    # Specify the desired load balancing algorithm
    virtual-server.f5.com/balance: "least-connections-node"
spec:
  rules:
  - host: mysite.example.com
    http:
      paths:
      - path: /app1
        backend:
          serviceName: myService1
          servicePort: 80
      - path: /app2
        backend:
          serviceName: myService2
          servicePort: 80

f5-k8s-ingress-fanout.yaml

Name-based virtual hosting

A Name-based virtual hosting ingress creates the following BIG-IP objects:

  • One (1) virtual server with one (1) pool for each Service.
  • Local traffic policies that route requests to specific pools based on host name and path.

Tip

If you don’t specify any hosts or paths, the BIG-IP device will proxy traffic for all hosts/paths for the Service specified in the backend section of the Ingress Resource.

Specific hosts
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: ing-virtual-hosting
 namespace: default
 annotations:
  # Provide an IP address for the BIG-IP virtual server;
  # Set to "controller-default" if you want to use the Controller's
  # "default-ingress-ip" (introduced in v1.4.0)
  virtual-server.f5.com/ip: "1.2.3.4"
  # Specify the BIG-IP partition where the Controller should create the virtual server.
  virtual-server.f5.com/partition: "k8s"
  # Specify the desired load balancing algorithm
  virtual-server.f5.com/balance: "least-connections-node"
  # Specify the port on which you want to handle requests
  virtual-server.f5.com/http-port: "80"
spec:
 rules:
 # URL
 - host: site1.example.com
   http:
     # path to Service from URL
     paths:
       - path: /app1
         backend:
           serviceName: myService1
           servicePort: 80
 # URL
 - host: site2.example.com
   http:
     # path to Service from URL
     paths:
       - path: /app2
         backend:
           serviceName: myService2
           servicePort: 80

f5-k8s-ingress-virtual-hosting.yaml

All hosts
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: ing-virtual-hosting
 namespace: default
 annotations:
  # Provide an IP address for the BIG-IP virtual server;
  # Set to "controller-default" if you want to use the Controller's
  # "default-ingress-ip" (introduced in v1.4.0)
  virtual-server.f5.com/ip: "1.2.3.4"
  # Specify the BIG-IP partition where the Controller should create the virtual server.
  virtual-server.f5.com/partition: "k8s"
  # Specify the desired load balancing algorithm
  virtual-server.f5.com/balance: "least-connections-node"
  # Specify the port on which you want to handle requests
  virtual-server.f5.com/http-port: "80"
spec:
 rules:
 # Omit the host name (URL) to match all hosts
 - http:
     # Provide the path to each Service you want to proxy
     paths:
     - path: /app1
       backend:
         serviceName: myService1
         servicePort: 80
     - path: /app2
       backend:
         serviceName: myService2
         servicePort: 80

f5-k8s-ingress-virtual-hosting_all.yaml

Upload the Ingress to the API server

If you have created a new Ingress resource, use the create command to upload it to the API server.

kubectl create -f myIngress.yaml

You can apply updates to an existing Ingress resource using the apply command.

kubectl apply -f myIngress.yaml

Verify object creation on the BIG-IP system

You can use TMOS or the BIG-IP configuration utility to verify that the BIG-IP Controller created the requested BIG-IP objects for your Ingress.

To verify using the BIG-IP configuration utility:

  1. Log in to the configuration utility at the management IP address (for example, https://10.190.25.225/tmui/login.jsp?).
  2. Select the correct partition from the Partition drop-down menu.
  3. Go to Local Traffic ‣ Virtual Servers to view all virtual servers, pools, and pool members.
  4. Go to Local Traffic ‣ Policies to view any new policies.

See the TMSH Reference Guide (PDF) for the relevant tmsh ltm commands.