OpenShift

What is Openshift?

RedHat OpenShift is the Kubernetes platform that provides a foundation for on-premises, hybrid, and multicloud deployments. With automated operations and streamlined life-cycle management, OpenShift empowers development teams to build and deploy new applications and helps operations teams provision, manage, and scale a Kubernetes platform.


How to deploy Openshift

Prerequisites

In order to deploy Openshift, you will need to use BIG-IP v13.1.0 or later. This is the minimum version that supports local/secondary IP addresses in the tunnel definition. If you are setting up for a lab/staging environment, see the CRC installation method here: https://code-ready.github.io/crc/

  • OCP 3.11 - one master and two worker nodes
  • CIS 2.0
  • AS3: 3.18+
  • BIG-IP 14.1.2.2
  • You will need to have the role of Administrator or Resource Administrator on the BIG-IP system.
  • Since CIS is using the AS3 declarative API, you need the AS3 extension installed on the BIG-IP. See the documentation here.

Quickstart: Standalone

Deploying CIS on OpenShift is different from Kubernetes. OpenShift uses OpenvSwitch (OVS) to connect pods locally, with VXLAN tunnels to connect different nodes. Openshift dynamically learns FDB of all nodes and ARP of individual app/nodes.

A high level example setup is shown below.

../../_images/openshift-quickstart-multinode.png
  1. Create a new OpenShift HostSubnet.

    Create a host subnet for the BIG-IP. This will provide the subnet for creating the tunnel self-IP.

    # oc create -f f5-kctlr-openshift-hostsubnet.yaml
    
    [root@ose-3-11-master openshift-3-11]# oc get hostsubnets
    NAME                               HOST                               HOST IP          SUBNET          EGRESS CIDRS   EGRESS IPS
    f5-server                          f5-server                          192.168.200.83   10.131.0.0/23        []          []
    ose-3-11-master.example.com        ose-3-11-master.example.com        192.168.200.84   10.128.0.0/23        []          []
    ose-3-11-node1.example.com         ose-3-11-node1.example.com         192.168.200.85   10.130.0.0/23        []          []
    ose-3-11-node2.lexample.com        ose-3-11-node2.example.com
    
  2. Create net tunnels vxlan vxlan-mp flooding-type multipoint.

    (tmos)# create net tunnels vxlan vxlan-mp flooding-type multipoint
    (tmos)# create net tunnels tunnel openshift_vxlan key 0 profile vxlan-mp local-address 192.168.200.83
    
  3. Add the BIG-IP device to the OpenShift overlay network.

    (tmos)# create net self 10.131.0.83/14 allow-service all vlan openshift_vxlan
    
  4. Create a new partition on your BIG-IP system. This needs to match the partition in the controller configuration.

    (tmos)# create auth partition openshift
    
  5. Create BIG-IP credentials and RBAC.

    Create BIG-IP credentials as Kubernetes secrets which you will use in CIS deployment.

    # oc create secret generic bigip-login --namespace kube-system --from-literal=username=<BIG-IP USERNAME> --from-literal=password=<BIG-IP PASSWORD>
    

    Create a Service account. A Service Account is required for containers running in a Pod to communicate with the API server of the Kubernetes/Openshift cluster.

    #  oc create serviceaccount bigip-ctlr -n kube-system
    
    (Service account "bigip-ctlr" will be created)
    

    Create Role Based Access Control (RBAC): refer to the RBAC YAML and apply to the OpenShift cluster.

  6. Create CIS deployment and deploy the controller.

    Refer to below CIS deployment and update accordingly as per the setup requirements. The Service account and BIG-IP credentials created above are used in CIS deployment. https://github.com/F5Networks/k8s-bigip-ctlr/blob/master/docs/_static/config_examples/openshift/f5-k8s-bigip-ctlr-openshift.yaml

  7. Cluster admin privileges to BIG-IP service account user:

    #  oc adm policy add-cluster-role-to-user cluster-admin -z bigip-ctlr -n kube-system
    
  8. Accessing AS3 logs from CIS.

    Verify that CIS POD is running. Execute the below command and make sure that CIS pod is running.

    #  oc get pods -n kube-system -owide
    

    Access AS3 specific logs from CIS logs.

    #  oc logs deploy/k8s-bigip-ctlr -n kube-system -f | grep -i "\[as3\]"
    

Quickstart: High Availability

You can use the BIG-IP Controller for OpenShift to manage a BIG-IP High Availability (HA) active-standby pair or device group (BIG-IP version 13.X+). You will need to deploy one BIG-IP Controller instance for each BIG-IP device. Each device will connect to a shared subnet that passes client traffic and to a management subnet.

../../_images/openshift-quickstart-ha.png

Complete the steps below to set up the solution shown in the diagram. Be sure to use the correct IP addresses and subnet masks for your OpenShift Cluster.


  1. Create a new OpenShift HostSubnet.

    Create one HostSubnet for each BIG-IP device. These will handle health monitor traffic. Also create one HostSubnet to pass data traffic using a floating IP address. Note that all files can be located under the bigip ha deployment folder. F5 recommends cloning this folder and making the modifications.

    oc create -f f5-kctlr-openshift-hostsubnet-ose-bigip-01.yaml
    oc create -f f5-kctlr-openshift-hostsubnet-ose-bigip-02.yaml
    oc create -f f5-kctlr-openshift-hostsubnet-float.yaml
    
    [root@ose-3-11-master bigip ha deployment]# oc get hostsubnets
    NAME                               HOST                               HOST IP          SUBNET          EGRESS CIDRS   EGRESS IPS
    f5-ose-bigip-01                    f5-ose-bigip-01                    192.168.200.82   10.131.0.0/23   []             []
    f5-ose-bigip-02                    f5-ose-bigip-02                    192.168.200.83   10.128.2.0/23   []             []
    f5-ose-float                       f5-ose-float                       192.168.200.81   10.129.2.0/23   []             []
    ose-3-11-master.lab.fp.f5net.com   ose-3-11-master.example.com        192.168.200.84   10.128.0.0/23   []             []
    ose-3-11-node1.lab.fp.f5net.com    ose-3-11-node1.example.com         192.168.200.85   10.130.0.0/23   []             []
    ose-3-11-node2.lab.fp.f5net.com    ose-3-11-node2.example.com         192.168.200.86   10.129.0.0/23   []             []
    
  2. Create a VXLAN profile that uses multi-cast flooding on each BIG-IP system.

    (tmos)# create net tunnels vxlan openshift_vxlan flooding-type multipoint
    
  3. Create a VXLAN tunnel:

    Set the key to 0 to grant the BIG-IP device access to all OpenShift projects and subnets.

    On ose-bigip-01 create the VXLAN tunnel:

    (tmos)# create /net tunnels tunnel openshift_vxlan key 0 profile openshift_vxlan local-address 192.168.200.81 secondary-address 192.168.200.82 traffic-group traffic-group-1
    

    On ose-bigip-02 create the VXLAN tunnel:

    (tmos)# create /net tunnels tunnel openshift_vxlan key 0 profile openshift_vxlan local-address 192.168.200.81 secondary-address 192.168.200.83 traffic-group traffic-group-1
    
  4. Create a Self IP in the VXLAN.

    Create a self IP address in the VXLAN on each device. The subnet mask you assign to the self IP must match the one that the OpenShift SDN assigns to nodes. Note that is /14 by default. Be sure to specify a floating traffic group (for example, traffic-group-1) otherwise the self IP will use the BIG-IP system default.

    [root@ose-3-11-master bigip ha deployment]# oc get hostsubnets
    NAME                               HOST                               HOST IP          SUBNET          EGRESS CIDRS   EGRESS IPS
    f5-ose-bigip-01                    f5-ose-bigip-01                    192.168.200.82   10.131.0.0/23   []             []
    f5-ose-bigip-02                    f5-ose-bigip-02                    192.168.200.83   10.128.2.0/23   []             []
    f5-ose-float                       f5-ose-float                       192.168.200.81   10.129.2.0/23   []             []
    

    On ose-bigip-01 create the self IP:

    (tmos)# create /net self 10.131.0.82/14 allow-service default vlan openshift_vxlan
    

    On ose-bigip-02 create the self IP:

    (tmos)# create /net self 10.128.2.83/14 allow-service default vlan openshift_vxlan
    

    On the active BIG-IP, create a floating IP address in the subnet assigned by the OpenShift SDN:

    (tmos)# create /net self 10.129.2.81/14 allow-service default traffic-group traffic-group-1 vlan openshift_vxlan
    
  5. Create a new partition on your BIG-IP system.

    (tmos)# create auth partition openshift
    
  6. Create BIG-IP credentials and RBAC.

    Create BIG-IP credentials as Kubernetes secrets which you will use in CIS deployment.

    # oc create secret generic bigip-login --namespace kube-system --from-literal=username=<BIG-IP USERNAME> --from-literal=password=<BIG-IP PASSWORD>
    

    Create a Service account. A Service Account is required for containers running in a Pod, to communicate with the API server of the Kubernetes/Openshift cluster.

    #  oc create serviceaccount bigip-ctlr -n kube-system
    

    Create Role Based Access Control (RBAC) by referring to this RBAC YAML and apply to the OpenShift cluster.

  7. Create CIS deployment and deploy the controller.

    Refer to below CIS deployment and update accordingly as per the setup requirements. The Service account and BIG-IP credentials created above are used in CIS deployment. https://github.com/F5Networks/k8s-bigip-ctlr/blob/master/docs/_static/config_examples/openshift/f5-k8s-bigip-ctlr-openshift.yaml

  8. Add Cluster admin privileges to BIG-IP service account user.

    #  oc adm policy add-cluster-role-to-user cluster-admin -z bigip-ctlr -n kube-system
    
  9. Access AS3 logs from CIS.

    Verify that CIS POD is running by running the following command:

    #  oc get pods -n kube-system -owide
    

    Access AS3 specific logs from CIS logs.

    #  oc logs deploy/k8s-bigip-ctlr -n kube-system -f | grep -i "\[as3\]"
    

Repository

See the repository on GitHub for more examples.

How to Deploy CIS Using OpenShift

Operator

An Operator is a method of packaging, deploying and managing a Kubernetes application. A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl/oc tooling. You can think of Operators as the runtime that manages this type of application on Kubernetes.

Conceptually, an Operator takes human operational knowledge and encodes it into software that is more easily packaged and shared with consumers.

The F5 BIG-IP CIS (k8s-bigip-ctlr) is a cloud-native connector that can use either Kubernetes or OpenShift as a BIG-IP orchestration platform.

F5 BIG-IP CIS Operator is a Service Operator which installs F5 BIG-IP CIS on OpenShift platforms 4.x.

Installing Operator

  1. Access OCP web console: From CLI, login as admin using CRC given credentials. In the example below, the username is kubeadmin. and password is db9Dr-J2csc-8oP78-9sbmf.

    $ eval $(crc  oc-env)
    $ oc login -u kubeadmin -p db9Dr-J2csc-8oP78-9sbmf https://api.crc.testing:6443
    
  2. Within the OCP web console, in the left Menu bar, click Operator Hub and search for “f5” to see the Certified F5 BIG-IP CIS Operator.

    ../../_images/ocp-bigip-controller.jpeg
  3. Click Install to go through the guided installation process.

    ../../_images/create-operator-subscription.jpeg

    When Operator is Subscribed, Operator is installed based on approval strategy. The default approval strategy is Automatic.

    • Manual: Requires administrator approval to install new updates.
    • Automatic: When a new release is available, updated automatic.

Configuring and Deploying CIS

  1. Under the Provided APIs column, click F5 BIG-IP Controller or F5BigIPCtlr to create an instance of F5 BIG-IP CIS.

    For example:

    apiVersion: cis.f5.com/v1
    kind: F5BigIpCtlr
    metadata:
    name: f5-server
    namespace: openshift-operators
    spec:
    args:
        manage_routes: true
        agent: as3
        log_level: DEBUG
        route_vserver_addr: 172.16.1.4
        bigip_partition: ocp
        openshift_sdn_name: /Common/openshift_vxlan
        bigip_url: 172.16.2.23
        log_as3_response: true
        insecure: true
        pool-member-type: cluster
    bigip_login_secret: f5-bigip-ctlr-login
    image:
        pullPolicy: Always
        repo: k8s-bigip-ctlr
        user: f5networks
    namespace: kube-system
    rbac:
        create: true
    resources: {}
    serviceAccount:
        create: true
    version: latest
    
  2. Verify the installation:

    $ oc get pods -n kube-system
    
    NAME                                       READY   STATUS    RESTARTS   AGE
    f5-server-f5-bigip-ctlr-7c77d6846f-z7bhp   1/1     Running   0          112s
    

Repository

See the repository for F5 BIG-IP CIS Operator on GitHub.


Note

To provide feedback on Container Ingress Services or this documentation, you can file a GitHub Issue.