F5 Solutions for Containers > Class 2: OpenShift with F5 Container Ingress Service > Module 2: CIS Using ClusterIP Mode Source | Edit on
Lab 2.1 - Install & Configure CIS in ClusterIP Mode¶
In the previous moudule we learned about Nodeport Mode. Here we’ll learn about ClusterIP Mode.
See also
For more information see BIG-IP Deployment Options
BIG-IP Setup¶
With ClusterIP we’re utilizing VXLAN to communicate with the application pods. To do so we’ll need to configure BIG-IP first.
If not already connected, RDP to the UDF lab “jumpbox” host. Otherwise resume previous session.
Open firefox and connect to bigip1. For your convenience there’s a shortcut on the toolbar. Username and password are: admin/admin
Attention
Be sure to be in the
Common
partition before creating the following objects.First we need to setup a partition that will be used by F5 Container Ingress Service.
Note
This step was performed in the previous module. Verify the “okd” partion exists and if not follow the instructions below.
- GoTo:
- Create a new partition called “okd” (use default settings)
- Click Finished
# From the CLI: ssh admin@10.1.1.4 tmsh create auth partition okd
Install AS3 via the management console
Attention
This has been done to save time. If needed see Module1 / Lab 1.1 / Install AS3 Steps
Create a vxlan tunnel profile
- GoTo:
- Create a new profile called “okd-vxlan”
- set Port = 4789
- Set the Flooding Type = Multipoint
- Click Finished
# From the CLI: ssh admin@10.1.1.4 tmsh create net tunnel vxlan okd-vxlan { app-service none port 4789 flooding-type multipoint }
Create a vxlan tunnel
- GoTo:
- Create a new tunnel called “okd-tunnel”
- Set the Profile to the one previously created called “okd-vxlan”
- set the key = 0
- Set the Local Address to 10.1.1.4
- Click Finished
# From the CLI: ssh admin@10.1.1.4 tmsh create net tunnel tunnel okd-tunnel { app-service none key 0 local-address 10.1.1.4 profile okd-vxlan }
Create the vxlan tunnel self-ip
Tip
For your SELF-IP subnet, remember it is a /14 and not a /23.
Why? The Self-IP has to know all other /23 subnets are local to this namespace, which includes Master1, Node1, Node2, etc. Each of which have their own /23.
Many students accidently use /23, doing so would limit the self-ip to only communicate with that subnet. When trying to ping services on other /23 subnets from the BIG-IP for instance, communication will fail as your self-ip doesn’t have the proper subnet mask to know the other subnets are local.
- GoTo:
- Create a new Self-IP called “okd-vxlan-selfip”
- Set the IP Address to “10.131.0.1”.
- Set the Netmask to “255.252.0.0”
- Set the VLAN / Tunnel to “okd-tunnel” (Created earlier)
- Set Port Lockdown to “Allow All”
- Click Finished
# From the CLI: ssh admin@10.1.1.4 tmsh create net self okd-vxlan-selfip { app-service none address 10.131.0.1/14 vlan okd-tunnel allow-service all }
CIS Deployment¶
Note
- For your convenience the file can be found in /home/ubuntu/agilitydocs/docs/class2/openshift (downloaded earlier in the clone git repo step).
- Or you can cut and paste the file below and create your own file.
- If you have issues with your yaml and syntax (indentation MATTERS), you can try to use an online parser to help you : Yaml parser
On the jumphost open a terminal and start an SSH session with okd-master1.
Note
This session should be up and running from the previous module.
# If directed to, accept the authenticity of the host by typing "yes" and hitting Enter to continue. ssh centos@okd-master1
Just like the previous module where we deployed CIS in NodePort mode we need to create a “secret”, “serviceaccount”, and “clusterrolebinding”.
Important
This step can be skipped if previously done in module1(NodePort). Some classes may choose to skip module1.
oc create secret generic bigip-login -n kube-system --from-literal=username=admin --from-literal=password=admin oc create serviceaccount k8s-bigip-ctlr -n kube-system oc create clusterrolebinding k8s-bigip-ctlr-clusteradmin --clusterrole=cluster-admin --serviceaccount=kube-system:k8s-bigip-ctlr
Next let’s explore the f5-hostsubnet.yaml file
cd ~/agilitydocs/docs/class2/openshift cat bigip-hostsubnet.yaml
You’ll see a config file similar to this:
bigip-hostsubnet.yaml¶1 2 3 4 5 6 7 8 9
apiVersion: v1 kind: HostSubnet metadata: name: openshift-f5-node annotations: pod.network.openshift.io/fixed-vnid-host: "0" host: openshift-f5-node hostIP: 10.1.1.4 subnet: "10.131.0.0/23"
Attention
This YAML file creates an OpenShift Node and the Host is the BIG-IP with an assigned /23 subnet of IP 10.131.0.0 (3 images down).
Next let’s look at the current cluster, you should see 3 members (1 master, 2 nodes)
oc get hostsubnet
Now create the connector to the BIG-IP device, then look before and after at the attached devices
oc create -f bigip-hostsubnet.yaml
You should see a successful creation of a new OpenShift Node.
At this point nothing has been done to the BIG-IP, this only was done in the OpenShift environment.
oc get hostsubnet
You should now see OpenShift configured to communicate with the BIG-IP
Important
The Subnet assignment, in this case is 10.131.0.0/23, was assigned by the subnet: “10.131.0.0/23” line in “HostSubnet” yaml file.
Note
In this lab we’re manually assigning a subnet. We have the option to let openshift auto assign ths by removing subnet: “10.131.0.0/23” line at the end of the “hostsubnet” yaml file and setting the assign-subnet: “true”. It would look like this:
apiVersion: v1 kind: HostSubnet metadata: name: openshift-f5-node annotations: pod.network.openshift.io/fixed-vnid-host: "0" pod.network.openshift.io/assign-subnet: "true" host: openshift-f5-node hostIP: 10.1.1.4
Now that we have added a HostSubnet for bigip1 we can launch the CIS deployment. It will start the f5-k8s-controller container on one of the worker nodes.
Attention
This may take around 30s to get to a running state.
cd ~/agilitydocs/docs/class2/openshift cat cluster-deployment.yaml
You’ll see a config file similar to this:
cluster-deployment.yaml¶1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
apiVersion: apps/v1 kind: Deployment metadata: name: k8s-bigip-ctlr namespace: kube-system spec: replicas: 1 selector: matchLabels: app: k8s-bigip-ctlr template: metadata: name: k8s-bigip-ctlr labels: app: k8s-bigip-ctlr spec: serviceAccountName: k8s-bigip-ctlr containers: - name: k8s-bigip-ctlr image: "f5networks/k8s-bigip-ctlr:2.4.1" imagePullPolicy: IfNotPresent env: - name: BIGIP_USERNAME valueFrom: secretKeyRef: name: bigip-login key: username - name: BIGIP_PASSWORD valueFrom: secretKeyRef: name: bigip-login key: password command: ["/app/bin/k8s-bigip-ctlr"] args: [ "--bigip-username=$(BIGIP_USERNAME)", "--bigip-password=$(BIGIP_PASSWORD)", "--bigip-url=https://10.1.1.4:8443", "--insecure=true", "--bigip-partition=okd", "--namespace=default", "--manage-routes=true", "--route-vserver-addr=10.1.1.4", "--route-http-vserver=okd_http_vs", "--route-https-vserver=okd_https_vs", "--route-label=hello-world", "--pool-member-type=cluster", "--openshift-sdn-name=/Common/okd-tunnel" ]
Create the CIS deployment with the following command
oc create -f cluster-deployment.yaml
Verify the deployment “deployed”
oc get deployment k8s-bigip-ctlr --namespace kube-system
To locate on which node CIS is running, you can use the following command:
oc get pods -o wide -n kube-system
We can see that our container, in this example, is running on okd-node1 below.
Troubleshooting¶
Check the container/pod logs via oc
command. You also have the option of
checking the Docker container as described in the previos module.
Using the full name of your pod as showed in the previous image run the following command:
# For example: oc logs k8s-bigip-ctlr-79b8f9cbd8-smsqs -n kube-system
Attention
You will see ERROR in this log output. These errors can be ignored. The lab will work as expected.