What is OpenShift?¶
RedHat OpenShift is the Kubernetes platform that provides a foundation for on-premises, hybrid, and multicloud deployments. With automated operations and streamlined life-cycle management, OpenShift empowers development teams to build and deploy new applications and helps operations teams provision, manage, and scale a Kubernetes platform.
How to deploy OpenShift¶
These are the mandatory requirements for deploying CIS:
OpenShift Cluster must be up and running
BIG-IP partition to create OpenShift cluster objects which can be created on the BIG-IP using the following tmos command:
create auth partition <cis_managed_partition>
You need a user with administrative access to this partition
If you need to pull the k8s-bigip-ctlr image from a private Doc.ker registry, store your Docker login credentials as a Secret.
Additionally, if you are deploying the CIS in Cluster Mode you need to have following prerequisites. For more information, see Deployment Options.
- You must have a fully active/licensed BIG-IP. SDN must be licensed. For more information, see BIG-IP VE license support for SDN services.
- VXLAN tunnel should be configured from OpenShift Cluster to BIG-IP. For more information see, Creating VXLAN Tunnels.
Installing CIS Manually¶
Add BIG-IP credentials as K8S secrets.
oc create secret generic bigip-login -n kube-system --from-literal=username=admin --from-literal=password=<password>
Create a service account for deploying CIS.
oc create serviceaccount k8s-bigip-ctlr -n kube-system
Create a Cluster Role and Cluster Role Binding on the Openshift Cluster using the examples below.
oc apply -f openshift_rbac.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
# For use in OpenShift clusters only apiVersion: v1 kind: ClusterRole metadata: annotations: authorization.openshift.io/system-only: "true" name: system:bigip-ctlr rules: - apiGroups: - "" - "extensions" resources: - nodes - services - endpoints - namespaces - ingresses - routes - secrets verbs: - get - list - watch - apiGroups: - "" - "extensions" resources: - configmaps - events - ingresses/status verbs: - get - list - watch - update - create - patch - apiGroups: - cis.f5.com resources: - virtualservers - tlsprofiles - transportservers verbs: - get - list - watch - update --- apiVersion: v1 kind: ClusterRoleBinding metadata: name: bigip-ctlr-role userNames: - system:serviceaccount:kube-system:bigip-ctlr subjects: - kind: ServiceAccount namespace: kube-system name: bigip-ctlr roleRef: name: system:bigip-ctlr
You can substitute a Role and RoleBinding if your Controller does not need access to the entire Cluster.
Create the Cluster admin privileges for the BIG-IP service account user with the following command:
oc adm policy add-cluster-role-to-user cluster-admin -z bigip-ctlr -n kube-system
Create a CIS deployment using cis_deploy.yaml as shown below:
oc apply -f cis_deploy.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
apiVersion: apps/v1 kind: Deployment metadata: name: k8s-bigip-ctlr-deployment namespace: kube-system spec: # DO NOT INCREASE REPLICA COUNT replicas: 1 selector: matchLabels: app: k8s-bigip-ctlr-deployment template: metadata: labels: app: k8s-bigip-ctlr-deployment spec: # Name of the Service Account bound to a Cluster Role with the required # permissions containers: - name: k8s-bigip-ctlr image: "f5networks/k8s-bigip-ctlr" env: - name: BIGIP_USERNAME valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: username - name: BIGIP_PASSWORD valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: password command: ["/app/bin/k8s-bigip-ctlr"] args: [ # See the k8s-bigip-ctlr documentation for information about # all config options # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest "--bigip-username=$(BIGIP_USERNAME)", "--bigip-password=$(BIGIP_PASSWORD)", "--bigip-url=<ip_address-or-hostname>", "--bigip-partition=<name_of_partition>", "--pool-member-type=nodeport", "--insecure", # for secure communication provide the internal ca certificates using config-map with below option and remove insecure parameter #"--trusted-certs-cfgmap=<namespace/configmap>", ] serviceAccount: bigip-controller serviceAccountName: bigip-controller imagePullSecrets: # Secret that gives access to a private docker registry #- name: f5-docker-images # Secret containing the BIG-IP system login credentials - name: bigip-login
Installing CIS using Operators on OpenShift Cluster¶
An Operator is a method of packaging, deploying, and managing a Kubernetes application. A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl/oc tooling. You can think of Operators as the runtime that manages this type of application on Kubernetes. Conceptually, an Operator takes human operational knowledge and encodes it into software that is more easily packaged and shared with consumers.
The F5 BIG-IP CIS (k8s-bigip-ctlr) is a cloud-native connector that can use either Kubernetes or OpenShift as a BIG-IP orchestration platform. F5 BIG-IP CIS Operator is a Service Operator which installs F5 BIG-IP CIS on OpenShift platforms 4.x.
Before you install CIS using Operators on OpenShift, you must create BIG-IP login credentials to use with Operator Helm Charts:
oc create secret generic <SECRET-NAME> -n kube-system --from-literal=username=<USERNAME> --from-literal=password=<PASSWORD>
Access the OCP web console: From CLI, login as admin using CRC given credentials. In the example below, the username is
kubeadminand the password is
$ eval $(crc oc-env) $ oc login -u kubeadmin -p db9Dr-J2csc-8oP78-9sbmf https://api.crc.testing:6443
Within the OCP web console, in the left Menu bar, click Operator Hub and search for “f5” to see the Certified F5 BIG-IP CIS Operator.
Click Install to go through the guided installation process.
When Operator is Subscribed, Operator is installed based on approval strategy. The default approval strategy is Automatic.
- Manual: Requires administrator approval to install new updates.
- Automatic: When a new release is available, updated automatic.
Creating VXLAN Tunnels on Openshift Cluster¶
Log in to the BIG-IP and create net tunnels vxlan vxlan-mp flooding-type multipoint on BIG-IP.
create net tunnels vxlan vxlan-mp flooding-type multipoint
create net tunnels tunnel openshift_vxlan key 0 profile vxlan-mp local-address 192.168.200.83
Add the BIG-IP device to the OpenShift overlay network.
create net self 10.131.0.83/14 allow-service all vlan openshift_vxlan
Create a new HostSubnet for BIG-IP on the OpenShift cluster. This will provide the subnet for creating the tunnel self-IP.
oc create -f openshift-hostsubnet.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14
apiVersion: v1 kind: HostSubnet metadata: name: openshfit-f5-node annotations: pod.network.openshift.io/fixed-vnid-host: "0" pod.network.openshift.io/assign-subnet: "true" # provide a name for the node that will serve as BIG-IP's entry into the cluster host: openshfit-f5-node # The hostIP address will be the BIG-IP interface address routable to the # OpenShift Origin nodes. # This address is the BIG-IP VTEP in the SDN's VXLAN. hostIP: 10.1.1.4 subnet: "10.131.0.0/23"
OpenShift Route Resources¶
You can use OpenShift Route resources in an existing deployment once you replace the OpenShift F5 Router with the BIG-IP Controller.
Supported Route Configurations¶
The CIS supports a single path-based route for TLS re-encryption. Multiple path-based routes are not currently supported.
|Type||Client Connection Encrypted||Server Connection Encrypted||Path Support||SSL Termination on BIG-IP||Description|
|Unsecured||No||No||Yes||No||The BIG-IP system forwards unsecured traffic from the client to the endpoint.|
The Controller maintains a new client SSL profile on the BIG-IP system based on the client certificate and key from the Route resource.
|Passthrough Terminated||Yes||Yes||No||No||The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.|
The Controller maintains a new BIG-IP client SSL profile based on the client certificate and key from the Route resource.
The Controller maintains a new BIG-IP server SSL profile based on the server CA certificate from the Route resource for re-encrypting the traffic.
The BIG-IP system uses an iRule to select the destination pool based on SNI and forward the re-encrypted traffic.
Supported Route Annotations¶
|virtual-server.f5.com/balance||string||Optional||Sets the load balancing mode.||round-robin||Any supported load balancing algorithm|
|virtual-server.f5.com/clientssl||string||Optional||The name of a pre-configured client ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate and key within the Route’s configuration.||N/A|
|virtual-server.f5.com/serverssl||string||Optional||The name of a pre-configured server ssl profile on the BIG-IP system. The controller uses this profile instead of the certificate within the Route’s configuration.||N/A|
|virtual-server.f5.com/health||JSON object||Optional||Defines a health monitor for the Route resource.||N/A|
|path||string||Required||The path for the Service specified in the Route resource.||N/A|
|interval||integer||Required||The interval at which to check the health of the virtual server.||N/A|
|timeout||integer||Required||Number of seconds before the check times out.||N/A|
|send||string||Required||The send string to set in the health monitor.||N/A|
|recv||string||Optional||String or RegEx pattern to match in first 5,120 bytes of backend response.||N/A|
|virtual-server.f5.com/secure-serverssl||boolean||Optional||Specify to validate the server-side SSL certificate of re-encrypt terminated routes.||false||“true”, “false”|
|virtual-server.f5.com/rewrite-app-root||string||Optional||Root path redirection for the application.||N/A|
|virtual-server.f5.com/rewrite-target-url||string||Optional||URL host, path, or host and path to be rewritten.||N/A|
|virtual-server.f5.com/allow-source-range||string||Optional||Comma-separated list of CIDR addresses to allow inbound to Route services.||N/A||
Comma-separated, CIDR formatted, IP addresses.
|virtual-server.f5.com/waf||string||Optional||The name of a pre-configured WAF Policy on the BIG-IP system.||N/A|
For edge (client) termination, a Route must include either the certificate/key literal information in the Route Spec, or the clientssl annotation. For re-encrypt (server) termination, a Route must include either the destinationCaCertificate literal information in the Route Spec, or the serverssl annotation, in addition to the edge rules listed previously. If you want to use the configuration parameters default-clientssl or default-serverssl profiles for a Route, then specify those profile names in the Route annotations in addition to the controller configuration.
To provide feedback on Container Ingress Services or this documentation, you can file a GitHub Issue.