OpenShift

What is OpenShift?

RedHat OpenShift is the Kubernetes platform that provides a foundation for on-premises, hybrid, and multicloud deployments. With automated operations and streamlined life-cycle management, OpenShift empowers development teams to build and deploy new applications and helps operations teams provision, manage, and scale a Kubernetes platform.


How to deploy OpenShift

Prerequisites

These are the mandatory requirements for deploying CIS:

  • OpenShift Cluster must be up and running

  • AS3: 3.18+

  • BIG-IP partition to create OpenShift cluster objects which can be created on the BIG-IP using the following tmos command:

    create auth partition <cis_managed_partition>
    
  • You need a user with administrative access to this partition

  • If you need to pull the k8s-bigip-ctlr image from a private Doc.ker registry, store your Docker login credentials as a Secret.

Additionally, if you are deploying the CIS in Cluster Mode you need to have following prerequisites. For more information, see Deployment Options.


Installing CIS Manually

  1. Add BIG-IP credentials as K8S secrets.

    oc create secret generic bigip-login -n kube-system --from-literal=username=admin --from-literal=password=<password>
    
  2. Create a service account for deploying CIS.

    oc create serviceaccount k8s-bigip-ctlr -n kube-system
    
  3. Create a Cluster Role and Cluster Role Binding on the Openshift Cluster using the examples below.

    oc apply -f openshift_rbac.yaml
    
    openshift_rbac.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
     # For use in OpenShift clusters only
     apiVersion: v1
     kind: ClusterRole
     metadata:
       annotations:
         authorization.openshift.io/system-only: "true"
       name: system:bigip-ctlr
     rules:
     - apiGroups:
       - ""
       - "extensions"
       resources:
       - nodes
       - services
       - endpoints
       - namespaces
       - ingresses
       - routes
       - secrets
       verbs:
       - get
       - list
       - watch
     - apiGroups:
       - ""
       - "extensions"
       resources:
       - configmaps
       - events
       - ingresses/status
       verbs:
       - get
       - list
       - watch
       - update
       - create
       - patch
     - apiGroups:
       - cis.f5.com
       resources:
       - virtualservers
       - tlsprofiles
       - transportservers
       verbs:
       - get
       - list
       - watch
       - update
    
     ---
    
     apiVersion: v1
     kind: ClusterRoleBinding
     metadata:
         name: bigip-ctlr-role
     userNames:
     - system:serviceaccount:kube-system:bigip-ctlr
     subjects:
     - kind: ServiceAccount
       namespace: kube-system
       name: bigip-ctlr
     roleRef:
       name: system:bigip-ctlr
    

    Important

    You can substitute a Role and RoleBinding if your Controller does not need access to the entire Cluster.

  4. Create the Cluster admin privileges for the BIG-IP service account user with the following command:

    oc adm policy add-cluster-role-to-user cluster-admin -z bigip-ctlr -n kube-system
    
  5. Create a CIS deployment using cis_deploy.yaml as shown below:

    oc apply -f cis_deploy.yaml
    
    cis_deploy.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: k8s-bigip-ctlr-deployment
      namespace: kube-system
    spec:
    # DO NOT INCREASE REPLICA COUNT
      replicas: 1
      selector:
        matchLabels:
          app: k8s-bigip-ctlr-deployment
      template:
        metadata:
          labels:
            app: k8s-bigip-ctlr-deployment
        spec:
          # Name of the Service Account bound to a Cluster Role with the required
          # permissions
          containers:
            - name: k8s-bigip-ctlr
              image: "f5networks/k8s-bigip-ctlr"
              env:
                - name: BIGIP_USERNAME
                  valueFrom:
                    secretKeyRef:
                    # Replace with the name of the Secret containing your login
                    # credentials
                      name: bigip-login
                      key: username
                - name: BIGIP_PASSWORD
                  valueFrom:
                    secretKeyRef:
                    # Replace with the name of the Secret containing your login
                    # credentials
                      name: bigip-login
                      key: password
              command: ["/app/bin/k8s-bigip-ctlr"]
              args: [
                # See the k8s-bigip-ctlr documentation for information about
                # all config options
                # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest
                "--bigip-username=$(BIGIP_USERNAME)",
                "--bigip-password=$(BIGIP_PASSWORD)",
                "--bigip-url=<ip_address-or-hostname>",
                "--bigip-partition=<name_of_partition>",
                "--pool-member-type=nodeport",
                "--insecure",
                # for secure communication provide the internal ca certificates using config-map with below option and remove insecure parameter
                #"--trusted-certs-cfgmap=<namespace/configmap>",
                ]
          serviceAccount: bigip-controller
          serviceAccountName: bigip-controller
        imagePullSecrets:
            # Secret that gives access to a private docker registry
            #- name: f5-docker-images
            # Secret containing the BIG-IP system login credentials
            - name: bigip-login
    

Installing CIS using Operators on OpenShift Cluster

An Operator is a method of packaging, deploying, and managing a Kubernetes application. A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl/oc tooling. You can think of Operators as the runtime that manages this type of application on Kubernetes. Conceptually, an Operator takes human operational knowledge and encodes it into software that is more easily packaged and shared with consumers.

The F5 BIG-IP CIS (k8s-bigip-ctlr) is a cloud-native connector that can use either Kubernetes or OpenShift as a BIG-IP orchestration platform. F5 BIG-IP CIS Operator is a Service Operator which installs F5 BIG-IP CIS on OpenShift platforms 4.x.

Before you install CIS using Operators on OpenShift, you must create BIG-IP login credentials to use with Operator Helm Charts:

oc create secret generic <SECRET-NAME> -n kube-system
--from-literal=username=<USERNAME> --from-literal=password=<PASSWORD>
  1. Access the OCP web console: From CLI, login as admin using CRC given credentials. In the example below, the username is kubeadmin and the password is db9Dr-J2csc-8oP78-9sbmf.

    $ eval $(crc  oc-env)
    $ oc login -u kubeadmin -p db9Dr-J2csc-8oP78-9sbmf https://api.crc.testing:6443
    
  2. Within the OCP web console, in the left Menu bar, click Operator Hub and search for “f5” to see the Certified F5 BIG-IP CIS Operator.

    ../../_images/ocp-bigip-controller.jpeg
  3. Click Install to go through the guided installation process.

    ../../_images/create-operator-subscription.jpeg

    When Operator is Subscribed, Operator is installed based on approval strategy. The default approval strategy is Automatic.

    • Manual: Requires administrator approval to install new updates.
    • Automatic: When a new release is available, updated automatic.

Creating VXLAN Tunnels on Openshift Cluster

  1. Log in to the BIG-IP and create net tunnels vxlan vxlan-mp flooding-type multipoint on BIG-IP.

    create net tunnels vxlan vxlan-mp flooding-type multipoint
    
    create net tunnels tunnel openshift_vxlan key 0 profile vxlan-mp local-address 192.168.200.83
    
  2. Add the BIG-IP device to the OpenShift overlay network.

    create net self 10.131.0.83/14 allow-service all vlan openshift_vxlan
    
  3. Create a new HostSubnet for BIG-IP on the OpenShift cluster. This will provide the subnet for creating the tunnel self-IP.

    oc create -f openshift-hostsubnet.yaml
    
openshift-hostsubnet.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
apiVersion: v1
kind: HostSubnet
metadata:
  name: openshfit-f5-node
  annotations:
    pod.network.openshift.io/fixed-vnid-host: "0"
    pod.network.openshift.io/assign-subnet: "true"
# provide a name for the node that will serve as BIG-IP's entry into the cluster
host: openshfit-f5-node
# The hostIP address will be the BIG-IP interface address routable to the
# OpenShift Origin nodes.
# This address is the BIG-IP VTEP in the SDN's VXLAN.
hostIP: 10.1.1.4
subnet: "10.131.0.0/23"
../../_images/f5-oc-hostsubnet2.png

Examples Repository

View more examples on GitHub.


OpenShift Route Resources

Note

You can use OpenShift Route resources in an existing deployment once you replace the OpenShift F5 Router with the BIG-IP Controller.

Supported Route Configurations

Important

The CIS supports a single path-based route for TLS re-encryption. Multiple path-based routes are not currently supported.

Type Client Connection Encrypted Server Connection Encrypted Path Support SSL Termination on BIG-IP Description
Unsecured No No Yes No The BIG-IP system forwards unsecured traffic from the client to the endpoint.
Edge Terminated Yes No Yes Yes

The Controller maintains a new client SSL profile on the BIG-IP system based on the client certificate and key from the Route resource.

  • Set insecureEdgeTerminationPolicy in the Route resource to Allow to enable support for insecure client connections.
  • Set insecureEdgeTerminationPolicy in the Route resource to Redirect to redirect HTTP client connections to the HTTPS endpoint.
Passthrough Terminated Yes Yes No No The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.
Re-encrypt Terminated Yes Yes Yes Yes

The Controller maintains a new BIG-IP client SSL profile based on the client certificate and key from the Route resource.

The Controller maintains a new BIG-IP server SSL profile based on the server CA certificate from the Route resource for re-encrypting the traffic.

The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.

Supported Route Annotations

Annotation Type Required Description Default Allowed Values
virtual-server.f5.com/balance string Optional Sets the load balancing mode. round-robin Any supported load balancing algorithm
virtual-server.f5.com/clientssl string Optional The name of a pre-configured client ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate and key within the Route’s configuration. N/A  
virtual-server.f5.com/serverssl string Optional The name of a pre-configured server ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate within the Route’s configuration. N/A  
virtual-server.f5.com/health JSON object Optional Defines a health monitor for the Route resource. N/A  
  path string Required The path for the Service specified in the Route resource. N/A  
  interval integer Required The interval at which to check the health of the virtual server. N/A  
  timeout integer Required Number of seconds before the check times out. N/A  
  send string Required The send string to set in the health monitor. N/A  
  recv string Optional String or RegEx pattern to match in first 5,120 bytes of backend response. N/A  
virtual-server.f5.com/secure-serverssl boolean Optional Specify to validate the server-side SSL certificate of re-encrypt terminated routes. false “true”, “false”
virtual-server.f5.com/rewrite-app-root string Optional Root path redirection for the application. N/A  
virtual-server.f5.com/rewrite-target-url string Optional URL host, path, or host and path to be rewritten. N/A  
virtual-server.f5.com/allow-source-range string Optional Comma-separated list of CIDR addresses to allow inbound to Route services. N/A

Comma-separated, CIDR formatted, IP addresses.

ex. 1.2.3.4/32,2.2.2.0/24

virtual-server.f5.com/waf string Optional The name of a pre-configured WAF Policy on the BIG-IP system. N/A  

Important

For edge (client) termination, a Route must include either the certificate/key literal information in the Route Spec, or the clientssl annotation. For re-encrypt (server) termination, a Route must include either the destinationCaCertificate literal information in the Route Spec, or the serverssl annotation, in addition to the edge rules listed previously. If you want to use the configuration parameters default-clientssl or default-serverssl profiles for a Route, then specify those profile names in the Route annotations in addition to the controller configuration.


Note

To provide feedback on Container Ingress Services or this documentation, you can file a GitHub Issue.