Kubernetes¶
How to deploy Kubernetes¶
Overview¶
CIS can be configured in multiple ways depending on the customer scenario. CIS can be deployed on Kubernetes and OpenShift platform. CIS installation may differ based on the resources (for example: ConfigMap, Ingress, Routes, and CRD) used by the customer to expose the Kubernetes services. CIS installation also depends on BIG-IP deployment (stand alone and HAproxy configuration) and Kubernetes cluster networking (Flannel/Calico).
Prerequisites¶
These are the mandatory requirements for deploying CIS:
Kubernetes Cluster must be up and running
AS3: 3.18+
BIG-IP partition to create Kubernetes cluster objects which can be created on the BIG-IP using the following tmos command:
create auth partition <cis_managed_partition>
You need a user with administrative access to this partition.
If you need to pull the k8s-bigip-ctlr image from a private Docker registry, store your Docker login credentials as a Secret.
Additionally, if you are deploying the CIS in Cluster Mode you need to have following prerequisites. For more information, see Deployment Options.
- You must have a fully active/licensed BIG-IP. SDN must be licensed. For more information, see BIG-IP VE license support for SDN services.
- VXLAN tunnel should be configured from Kubernetes Cluster to BIG-IP. For more information see, Creating VXLAN Tunnels on Kubernetes Cluster.
Installing CIS Manually¶
Add BIG-IP credentials as K8S secrets.
kubectl create secret generic bigip-login -n kube-system --from-literal=username=admin --from-literal=password=<password>
Create a service account for deploying CIS. In the example below, the Service Account is named
bigip-ctlr
.kubectl create serviceaccount bigip-ctlr -n kube-system
Create a Cluster Role and Cluster Role Binding on the Kubernetes Cluster using the examples below.
kubectl apply -f k8s_rbac.yaml
The example below shows the broadest supported permission set. You can narrow the permissions down to specific resources, namespaces, etc. to suit your needs. See the Kubernetes RBAC documentation for more information.
k8s_rbac.yaml¶1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
# for use in k8s clusters only kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: bigip-ctlr-clusterrole rules: - apiGroups: ["", "extensions"] resources: ["nodes", "services", "endpoints", "namespaces", "ingresses", "pods"] verbs: ["get", "list", "watch"] - apiGroups: ["", "extensions"] resources: ["configmaps", "events", "ingresses/status"] verbs: ["get", "list", "watch", "update", "create", "patch"] - apiGroups: ["", "extensions"] resources: ["secrets"] resourceNames: ["<secret-containing-bigip-login>"] verbs: ["get", "list", "watch"] - apiGroups: ["cis.f5.com"] resources: ["virtualservers", "tlsprofiles", "transportservers"] verbs: ["get", "list", "watch", "update"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: bigip-ctlr-clusterrole-binding namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: bigip-ctlr-clusterrole subjects: - apiGroup: "" kind: ServiceAccount name: bigip-ctlr namespace: kube-system
Create a CIS deployment using cis_deploy.yaml as shown below:
kubectl apply -f cis_deploy.yaml
cis_deploy.yaml¶1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
apiVersion: apps/v1 kind: Deployment metadata: name: k8s-bigip-ctlr-deployment namespace: kube-system spec: # DO NOT INCREASE REPLICA COUNT replicas: 1 selector: matchLabels: app: k8s-bigip-ctlr-deployment template: metadata: labels: app: k8s-bigip-ctlr-deployment spec: # Name of the Service Account bound to a Cluster Role with the required # permissions containers: - name: k8s-bigip-ctlr image: "f5networks/k8s-bigip-ctlr" env: - name: BIGIP_USERNAME valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: username - name: BIGIP_PASSWORD valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: password command: ["/app/bin/k8s-bigip-ctlr"] args: [ # See the k8s-bigip-ctlr documentation for information about # all config options # https://clouddocs.f5.com/containers/latest/ "--bigip-username=$(BIGIP_USERNAME)", "--bigip-password=$(BIGIP_PASSWORD)", "--bigip-url=<ip_address-or-hostname>", "--bigip-partition=<name_of_partition>", "--pool-member-type=nodeport", "--insecure", # for secure communication provide the internal ca certificates using config-map with below option and remove insecure parameter #"--trusted-certs-cfgmap=<namespace/configmap>", ] serviceAccount: bigip-ctlr serviceAccountName: bigip-ctlr imagePullSecrets: # Secret that gives access to a private docker registry #- name: f5-docker-images # Secret containing the BIG-IP system login credentials - name: bigip-login
Creating VXLAN Tunnels on Kubernetes Cluster¶
Log in to BIG-IP and create a VXLAN tunnel profile. Create a partition using the command below. In this example, the partition is named ‘kubernetes’.
tmsh create auth partition kubernetes
Create a VXLAN tunnel.
tmsh create net tunnels tunnel k8s-tunnel { app-service none key 1 local-address 10.1.1.4 profile k8s-vxlan }
Create the VXLAN tunnel self-IP.
tmsh create net self k8s-vxlan-selfip { address 10.244.20.1/16 vlan k8s-tunnel allow-service all }
Save the configuration.
tmsh save sys config
Before deploying CIS in ClusterIP mode, you need to configure BIG-IP as a node in the Kubernetes cluster. To do so you will need to modify
bigip-node.yaml
with the MAC address auto-created from the previous steps. From the jumpbox terminal, run the following command atbigip1
. Copy the displayed MAC Address.tmsh show net tunnels tunnel k8s-tunnel all-properties
Update the MAC address obtained in the previous step to following yaml file:
bigip-node.yaml¶1 2 3 4 5 6 7 8 9 10 11 12 13 14
apiVersion: v1 kind: Node metadata: name: bigip1 annotations: #Replace IP with Self-IP for your deployment flannel.alpha.coreos.com/public-ip: "10.1.1.4" #Replace MAC with your BIG-IP Flannel VXLAN Tunnel MAC flannel.alpha.coreos.com/backend-data: '{"VtepMAC":"2c:c2:60:23:0c:58"}' flannel.alpha.coreos.com/backend-type: "vxlan" flannel.alpha.coreos.com/kube-subnet-manager: "true" spec: #Replace Subnet with your BIG-IP Flannel Subnet podCIDR: "10.244.20.0/24"
Create the BIG-IP node:
kubectl create -f bigip-node.yaml
Verify “bigip1” node is created:
kubectl get nodes
Examples Repository¶
Note
To provide feedback on Container Ingress Services or this documentation, you can file a GitHub Issue.