Custom Resource Definitions¶
This page describes CIS in CRD Mode. This is an Alpha release that supports limited features.
What are CRDs?¶
- Custom resources are extensions of the Kubernetes API.
- A resource is an endpoint in the Kubernetes API that stores a collection of API objects. For example, the built-in pods resource contains a collection of Pod objects.
- A custom resource is an extension of the Kubernetes API that is not necessarily available in a default Kubernetes installation. It represents a customization of a particular Kubernetes installation. However, many core Kubernetes functions are now built using custom resources, making Kubernetes more modular.
- Custom resources can appear and disappear in a running cluster through dynamic registration, and cluster admins can update custom resources independently of the cluster itself. Once a custom resource is installed, users can create and access its objects using kubectl, just as they do for built-in resources like Pods.
- CIS supports 2 Custom Resources:
VirtualServer resource defines load balancing configuration for a domain name.
1 2 3 4 5 6 7 8 9 10 11 12 13
apiVersion: "cis.f5.com/v1" kind: VirtualServer metadata: name: coffee-virtual-server labels: f5cr: "true" spec: host: coffee.example.com virtualServerAddress: "172.16.3.4" pools: - path: /coffee service: svc-2 servicePort: 80
CIS will only process custom resources with
f5cr Label as
labels: f5cr: "true"
The above VirtualServer is insecure. Attach a TLSProfile to make it secure.
TLSProfile is used to specify the TLS termination for a single/list of services in a VirtualServer Custom Resource. TLS termination relies on SNI. Any non-SNI traffic received on port 443 may result in connection issues. TLSProfile can be created either with certificates stored as k8s secrets or can be referenced by profiles existing in BIG-IP.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
apiVersion: cis.f5.com/v1 kind: TLSProfile metadata: name: reencrypt-tls labels: f5cr: "true" spec: tls: termination: reencrypt clientSSL: /common/clientssl serverSSL: /common/serverssl reference: bigip # --> reference profiles created in BIG-IP by User hosts: - coffee.example.com
VirtualServer with TLSProfile¶
VirtualServer with TLSProfile is used to specify the TLS termination. TLS termination relies on SNI. Any non-SNI traffic received on port 443 may result in connection issues. The example below shows how to attach a TLSProfile to a VirtualServer.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
apiVersion: cis.f5.com/v1 kind: VirtualServer metadata: name: coffee-virtual-server labels: f5cr: "true" namespace: default spec: host: coffee.example.com tlsProfileName: reencrypt-tls. # --> This will attach reencrypt-tls TLSProfile virtualServerAddress: "172.16.3.4" pools: - path: /coffee service: svc servicePort: 80
- CIS has a 1:1 mapping for a domain (CommonName) and BIG-IP-VirtualServer.
- You can create any number of custom resources for a single domain. For example, you can create 2 VirtualServers with different terminations (for the same domain): one with edge and another with re-encrypt. To do this you need to create two VirtualServers: one with edge TLSProfile and another with re-encrypt TLSProfile. Both the VirutalServers should be created with same virtualServerAddress.
- Single or Group of VirtualServers (with the same virtualServerAddress) will be created as one common BIG-IP-VirtualServer.
- If you want to update secure virtual (TLS Virtual) server to insecure virtual (non-TLS server) server. You need to delete the secure virtual server first and create a new virtual server.
How CIS works with CRDs¶
- CIS registers to the Kubernetes client-go using informers to retrieve Virtual Server, TLSProfile, Service, Endpoint and Node create, update and delete events. Resources identified from such events will be pushed to a Resource Queue maintained by CIS.
- Resource Queue holds the resources to be processed.
- Virtual Server is the Primary citizen. Any changes in TLSProfile, Service, Endpoint, Node will process their affected Virtual Servers. For example, If svc-a is part of foo-VirtualServer and bar-VirtualServer, any changes in svc-a will put foo-VirtualServer and bar-VirtualServer in resource queue.
- Worker fetches the affected Virtual Servers from Resource Queue to populate a common structure which holds the configuration of all the Virtual Servers such as TLSProfile, Virtual Server IP, Pool Members and L7 LTM policy actions.
- VXLAN Manager prepares the BIG-IP NET configuration as AS3 cannot process FDB and ARP entries.
- LTM Configuration (using AS3) and NET Configuration (using CCCL) will be created in CIS Managed Partition defined by the user.
|pools||List of pool||Required||N/A||List of BIG-IP Pool members|
|virtualServerAddress||String||Required||N/A||IP Address of BIG-IP Virtual Server|
|TLSProfile||String||Required||N/A||Describes the TLS configuration for BIG-IP Virtual Server|
|path||String||Required||N/A||Path to access the service.|
|service||String||Required||N/A||Service deployed in Kubernetes cluster|
|nodeMemberLabel||String||Optional||N/A||List of Nodes to consider in NodePort mode as BIG-IP pool members. This option is only applicable for NodePort mode.|
|servicePort||String||Required||N/A||Port to access service|
|monitor||String||Optional||N/A||Health Monitor to check the health of Pool Members|
|type||String||Required||N/A||http or https|
||HTTP request string to send|
|recv||String||Optional||N/A||String or RegEx pattern to match in first 5,120 bytes of backend response.|
|interval||Int||Required||5||Seconds between health queries|
|timeout||Int||Optional||16||Seconds before query fails|
Health Monitor associated with the first path will be considere if multiple path has same backend
|termination||String||Required||N/A||Termination on BIG-IP Virtual Server. Allowed options are [edge, reencrypt, passthrough]|
|clientSSL||String||Required||N/A||ClientSSL Profile on the BIG-IP. Example /Common/clientssl|
|serverSSL||String||Optional||N/A||ServerSSL Profile on the BIG-IP. Example /Common/serverssl|
|reference||String||Required||N/A||Describes the location of profile, BIG-IP or k8s Secrets. We currently support BIG-IP profiles only|
Since CIS 2.0 uses the AS3 declarative API, you will need to have the AS3 extension version 3.18 or newer installed on BIG-IP. See instructions for installing AS3 on BIG-IP.
Create CIS Controller, BIG-IP Credentials, and RBAC Authentication¶
Install F5 CRDs. Download this YAML file and run the following commands.
kubectl create -f customresourcedefinitions.yml [-n kube-system]
Create BIG-IP Credentials:
kubectl create secret generic bigip-login -n kube-system --from-literal=username=admin --from-literal=password=dummy
Create Service Account
kubectl create serviceaccount bigip-ctlr [-n kube-system]
Create Cluster Role and Cluster Role Binding. Download this YAML file and run the following command.
kubectl create -f clusterrole.yml [-n kube-system]
Add BIG-IP device to VXLAN
- Overview of CIS VXLAN
- Configure VXLAN with CIS
- Deploy k8s-bigip-ctlr in cluster and customresource mode. Download this YAML file and run the following command.
kubectl create -f sample-cluster-k8s-bigip-ctlr-crd-secret.yml [-n kube-system]