Last updated on: 2023-09-12 04:11:59.

Kubernetes

CIS Installation

Overview

CIS can be configured in multiple ways depending on the customer scenario. CIS can be deployed on Kubernetes platform. CIS installation may differ based on the resources (for example: ConfigMap, Ingress, Routes, and CRD) used by the customer to expose the Kubernetes services. CIS installation also depends on BIG-IP deployment (Standalone and High Availability configuration) and Kubernetes cluster networking (Flannel/Calico).

Prerequisites

See also

Installing Kubernetes with Minikube.

These are the mandatory requirements for deploying CIS:

  • Kubernetes Cluster must be up and running.

  • AS3: 3.18+ must be installed on your BIG-IP system.

  • Use the latest TLS version and cipher suites in Kubernetes for kube-api.

  • Create a BIG-IP partition to manage Kubernetes objects. This partition can be created either via the GUI (System > Users > Partition List) or via our TMOS CLI:

    create auth partition <cis_managed_partition>

  • You need a user with administrative access to this partition.

  • If you need to pull the k8s-bigip-ctlr image from a private Docker registry, store your Docker login credentials as a Secret.

Additionally, if you are deploying CIS in Cluster Mode you need to have the following prerequisites. For more information, see Deployment Options.

Installing CIS Manually

  1. Add BIG-IP credentials as K8S secrets.

    kubectl create secret generic f5-bigip-ctlr-login -n kube-system --from-literal=username=admin --from-literal=password=<password>

  2. Create a service account for deploying CIS. In the example below, the Service Account is named bigip-ctlr.

    kubectl create serviceaccount bigip-ctlr -n kube-system

  3. Create a Cluster Role and Cluster Role Binding on the Kubernetes Cluster using the examples below.

    The example below shows the broadest supported permission set. You can narrow the permissions down to specific resources, namespaces, etc. to suit your needs. See the Kubernetes RBAC documentation for more information.

    k8s_rbac.yaml
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
     # for use in k8s clusters only
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: bigip-ctlr-clusterrole
 rules:
 - apiGroups: ["", "extensions", "networking.k8s.io"]
   resources: ["nodes", "services", "endpoints", "namespaces", "ingresses", "pods", "ingressclasses"]
   verbs: ["get", "list", "watch"]
 - apiGroups: ["", "extensions", "networking.k8s.io"]
   resources: ["configmaps", "events", "ingresses/status", "services/status"]
   verbs: ["get", "list", "watch", "update", "create", "patch"]
 - apiGroups: ["cis.f5.com"]
   resources: ["virtualservers","virtualservers/status", "tlsprofiles", "transportservers", "ingresslinks", "externaldns"]
   verbs: ["get", "list", "watch", "update", "patch"]
 - apiGroups: ["fic.f5.com"]
   resources: ["f5ipams", "f5ipams/status"]
   verbs: ["get", "list", "watch", "update", "create", "patch", "delete"]
 - apiGroups: ["apiextensions.k8s.io"]
   resources: ["customresourcedefinitions"]
   verbs: ["get", "list", "watch", "update", "create", "patch"]
 - apiGroups: ["", "extensions"]
   resources: ["secrets"]
   verbs: ["get", "list", "watch"]


 ---

 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
   name: bigip-ctlr-clusterrole-binding
   namespace: kube-system
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: bigip-ctlr-clusterrole
 subjects:
 - apiGroup: ""
   kind: ServiceAccount
   name: bigip-ctlr
   namespace: kube-system

    Push this configuration with the command:

    kubectl apply -f  k8s_rbac.yaml

    Important

    You can substitute a Role and RoleBinding if your Controller does not need access to the entire Cluster.

  4. Optionally, if you plan to use CIS in CRD mode, install Custom Resource Definitions. To install F5 CRDs, run the following commands:

    export CIS_VERSION=<cis-version>
# For example
# export CIS_VERSION=v2.12.0
kubectl create -f https://raw.githubusercontent.com/F5Networks/k8s-bigip-ctlr/${CIS_VERSION}/docs/config_examples/customResourceDefinitions/customresourcedefinitions.yml

  5. Create a CIS deployment using cis_deploy.yaml as shown below. You need to update the example file with the correct information, for example: the name of your BIG-IP secret, BIG-IP IP, BIG-IP partion, etc.

    cis_deploy.yaml
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
    apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-bigip-ctlr-deployment
  namespace: kube-system
spec:
# DO NOT INCREASE REPLICA COUNT
  replicas: 1
  selector:
    matchLabels:
      app: k8s-bigip-ctlr-deployment
  template:
    metadata:
      labels:
        app: k8s-bigip-ctlr-deployment
    spec:
      # Name of the Service Account bound to a Cluster Role with the required
      # permissions
      containers:
        - name: k8s-bigip-ctlr
          image: "f5networks/k8s-bigip-ctlr:latest"
          env:
            - name: BIGIP_USERNAME
              valueFrom:
                secretKeyRef:
                # Replace with the name of the Secret containing your login
                # credentials
                  name: f5-bigip-ctlr-login
                  key: username
            - name: BIGIP_PASSWORD
              valueFrom:
                secretKeyRef:
                # Replace with the name of the Secret containing your login
                # credentials
                  name: f5-bigip-ctlr-login
                  key: password
          command: ["/app/bin/k8s-bigip-ctlr"]
          args: [
            # See the k8s-bigip-ctlr documentation for information about
            # all config options
            # When insecure=true, this enables insecure SSL communication to the BIG-IP system."
            # https://clouddocs.f5.com/containers/latest/
            "--bigip-username=$(BIGIP_USERNAME)",
            "--bigip-password=$(BIGIP_PASSWORD)",
            "--bigip-url=<ip_address-or-hostname>",
            "--bigip-partition=<name_of_partition>",
            "--pool-member-type=nodeport",
            "--insecure",
            ]
      serviceAccountName: bigip-ctlr

    Push this configuration with the following command:

    kubectl apply -f  cis_deploy.yaml

Creating VXLAN Tunnels on Kubernetes Cluster (Optional)

This section is required only if you plan to use CIS in a ClusterIP Deployment. See Deployment Options for more information.

  1. Create a VXLAN tunnel. In the example below, the tunnel name is fl-vxlan.

    tmsh create net tunnels vxlan fl-vxlan port 8472 flooding-type none
    ../../_images/kubernetes-vxlan-1.png

  2. Create a VXLAN profile.

    tmsh create net tunnels tunnel fl-vxlan key 1 profile fl-vxlan local-address 192.168.200.91
    ../../_images/kubernetes-vxlan-2.png

  3. Create the VXLAN tunnel self IP.

    tmsh create net self 10.244.20.91 address 10.244.20.91/255.255.0.0 allow-service none vlan fl-vxlan
    ../../_images/kubernetes-vxlan-3.png

Installing CIS Using Helm Charts

See also

Introduction to Helm.

This is the simplest way to install CIS on a Kubernetes cluster. Helm is a package manager for Kubernetes. Helm is Kubernetes version of YUM or APT. Helm deploys something called charts, which you can think of as a packaged application. It is a collection of all your versioned, pre-configured application resources which can be deployed as one unit.

  1. Optionally, add BIG-IP credentials as K8S secrets.

    For Kubernetes, use the following command:

    kubectl create secret generic f5-bigip-ctlr-login -n kube-system --from-literal=username=admin --from-literal=password=<password>

  2. Add the CIS chart repository in Helm using following command:

    helm repo add f5-stable https://f5networks.github.io/charts/stable

  3. Create values.yaml as shown below:

    values.yaml
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
    bigip_login_secret: f5-bigip-ctlr-login
bigip_secret:
  create: false
  username:
  password:
rbac:
  create: true
serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: k8s-bigip-ctlr
 # This namespace is where the Controller lives;
namespace: kube-system
ingressClass:
  create: true
  ingressClassName: f5
  isDefaultIngressController: true
args:
  # See https://clouddocs.f5.com/containers/latest/userguide/config-parameters.html
  # NOTE: helm has difficulty with values using `-`; `_` are used for naming
  # and are replaced with `-` during rendering.
  # REQUIRED Params
  bigip_url: <ip_address-or-hostname>
  bigip_partition: <name_of_partition>
  # OPTIONAL PARAMS -- uncomment and provide values for those you wish to use.
  # verify_interval:
  # node-poll_interval:
  # log_level:
  # python_basedir: ~
  # VXLAN
  # openshift_sdn_name:
  # flannel_name:
  # KUBERNETES
  # default_ingress_ip:
  # kubeconfig:
  # namespaces: ["foo", "bar"]
  # namespace_label:
  # node_label_selector:
  # pool_member_type: nodeport
  # resolve_ingress_names:
  # running_in_cluster:
  # use_node_internal:
  # use_secrets:
  # insecure: true
  # custom-resource-mode: true
  # log-as3-response: true
  # gtm-bigip-password
  # gtm-bigip-url
  # gtm-bigip-username
  # ipam : true
image:
  # Use the tag to target a specific version of the Controller
  user: f5networks
  repo: k8s-bigip-ctlr
  pullPolicy: Always
version: latest
# affinity:
#   nodeAffinity:
#     requiredDuringSchedulingIgnoredDuringExecution:
#       nodeSelectorTerms:
#       - matchExpressions:
#         - key: kubernetes.io/arch
#           operator: Exists
# securityContext:
#   runAsUser: 1000
#   runAsGroup: 3000
#   fsGroup: 2000
# If you want to specify resources, uncomment the following
# limits_cpu: 100m
# limits_memory: 512Mi
# requests_cpu: 100m
# requests_memory: 512Mi
# Set podSecurityContext for Pod Security Admission and Pod Security Standards
# podSecurityContext:
#   runAsUser: 1000
#   runAsGroup: 1000
#   privileged: true

  1. Installing Helm charts:

    1. Install the Helm chart using the following command if BIG-IP credential secrets are created manually:

      helm install -f values.yaml <new-chart-name> f5-stable/f5-bigip-ctlr

    2. Install the Helm chart with --skip crds if BIG-IP credential secrets are created manually (without Custom Resource Definitions installations):

      helm install --skip-crds -f values.yaml <new-chart-name> f5-stable/f5-bigip-ctlr

    3. If you want to create the BIG-IP credential secret with Helm charts, use the following command:

      helm install --set bigip_secret.create="true" --set bigip_secret.username=$BIGIP_USERNAME --set bigip_secret.password=$BIGIP_PASSWORD -f values.yaml <new-chart-name> f5-stable/f5-bigip-ctlr

    Note

    For Kubernetes versions lower than 1.18, please use Helm chart version 0.0.14 as follows: helm install --skip-crds -f values.yaml <new-chart-name> f5-stable/f5-bigip-ctlr --version 0.0.14.


Chart parameters

Parameter Required Default Description
bigip_login_secret Optional f5-bigip-ctlr-login Secret that contains BIG-IP login credentials.
bigip_secret.create Optional false Create Kubernetes secret using username and password.
bigip_secret.username Optional N/A BIG-IP username to create the Kubernetes secret.
bigip_secret.password Optional N/A BIG-IP password to create the Kubernetes secret.
args.bigip_url Required N/A The management IP for your BIG-IP device.
args.bigip_partition Required f5-bigip-ctlr BIG-IP partition the CIS Controller will manage.
args.namespaces Optional N/A List of Kubernetes namespaces which CIS will monitor.
rbac.create Optional true Create ClusterRole and ClusterRoleBinding.
serviceAccount.name Optional f5-bigip-ctlr- serviceaccount Name of the ServiceAccount for CIS controller.
serviceAccount.create Optional true Create service account for the CIS controller.
namespace Optional kube-system Name of namespace CIS will use to create deployment and other resources.
image.user Optional f5networks CIS Controller image repository username.
image.repo Optional k8s-bigip-ctlr CIS Controller image repository name.
image.pullPolicy Optional Always CIS Controller image pull policy.
image.pullSecrets Optional N/A List of secrets of container registry to pull image.
version Optional latest CIS Controller image tag.
nodeSelector Optional N/A Dictionary of Node selector labels.
tolerations Optional N/A Array of labels.
limits_cpu Optional 100m CPU limits for the pod.
limits_memory Optional 512Mi Memory limits for the pod.
requests_cpu Optional 100m CPU request for the pod.
requests_memory Optional 512Mi Memory request for the pod.
affinity Optional N/A Dictionary of affinity.
securityContext Optional N/A Dictionary of deployment securityContext.
podSecurityContext Optional N/A Dictionary of pod securityContext.
ingressClass.ingressClassName Optional f5 Name of ingress class.
ingressClass.isDefaultIngressController Optional false CIS will monitor all the ingress resources if set true.
ingressClass.create Optional true Create ingress class.

Note

The parameters bigip_login_secret and bigip_secret are mutually exclusive. If both are defined in the values.yaml file, bigip_secret will be given priority.


Uninstalling Helm Chart

Run the following command to uninstall the chart.

helm del <new-chart>

Examples Repository

View more examples on GitHub.


Note

To provide feedback on Container Ingress Services or this documentation, please file a GitHub Issue.