F5 IPAM Controller

The F5 IPAM Controller (FIC) is a Docker container that runs in a container environment. It allocates IP addresses from an IPAM system’s address pool for hostnames in an orchestration environment. The F5 IPAM Controller watches orchestration-specific resources and consumes the hostnames within each resource.

The Controller can:

Allocate IP address from static IP address pool based on the CIDR mentioned in a Kubernetes resource The idea here is that we will support CRD, Type LB and probably also in the future route/ingress. We should make it more generic so that we don’t have to update this later, F5 IPAM Controller decides to allocate the IP from the respective IP address pool for the hostname specified in the virtualserver custom resource. Supported kubernetes resource :

IPAM Compatibility Matrix

Resource Minimum Version Supported
VS CRD CIS v2.2.2

Note

You must run BIG-IP Container Ingress Services in CRD Mode:

  • Use --custom-resource-mode=true in your CIS deployment to enable Custom Resource Mode.
  • Reminder:
    • CIS does not watch for Ingress/Routes/ConfigMaps when deployed in CRD Mode.
    • CIS does not support the combination of CRDs with any of Ingress/Routes and ConfigMaps.

Architectural diagram of how F5-IPAM-Controller (FIC) fits in the environment:

../../_images/ipam-1.png



Flow Chart for CIS-FIC:

../../_images/ipam-2.png

Installing IPAM

  1. --orchestration=kubernetes
    
  2. --ip-range='{"Dev":"172.16.3.21-172.16.3.30","Test":"172.16.3.31-172.16.3.40", "Production":"172.16.3.41-172.16.3.50","Default":"172.16.3.51-172.16.3.60"}'
    
  3. --log-level=debug
    
Parameter Type Required Default Description
orchestration String Required N/A Holds the orchestration environment. For example, Kubernetes.
ip-range String Required N/A The IP address ranges and from this range, it creates a pool of IP address range which gets allocated to the corresponding hostname in the virtual server CRD.
log-level String Optional N/A Log level parameter specifies various logging levels such as DEBUG, INFO, WARNING, ERROR, CRITICAL.
  1. Below is the RBAC for F5 IPAM Controller

    f5-ipam-rbac.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: ipam-ctlr-clusterrole
    rules:
      - apiGroups: ["fic.f5.com"]
        resources: ["f5ipams"]
        verbs: ["get", "list", "watch", "update", "patch"]
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: ipam-ctlr-clusterrole-binding
      namespace: kube-system
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ipam-ctlr-clusterrole
    subjects:
      - apiGroup: ""
        kind: ServiceAccount
        name: ipam-ctlr
        namespace: kube-system
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: ipam-ctlr
      namespace: kube-system
    

    f5-ipam-rbac.yaml

    Push this configuration with the following command:

    kubectl create -f f5-ipam-rbac.yaml
    
  2. Below is the CIS Deployment:

    cis-deployment-ipam-enabled.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: k8s-bigip-ctlr-deployment
      namespace: kube-system
    spec:
      replicas: 1
      template:
        metadata:
          name: k8s-bigip-ctlr
          labels:
            app: k8s-bigip-ctlr
        spec:
          serviceAccountName: bigip-ctlr
          containers:
            - name: k8s-bigip-ctlr
              image: "f5networks/k8s-bigip-ctlr"
              command: ["/app/bin/k8s-bigip-ctlr"]
              args: [
                "--bigip-username=$(BIGIP_USERNAME)",
                "--bigip-password=$(BIGIP_PASSWORD)",
                "--bigip-url=<ip_address-or-hostname>",
                "--bigip-partition=<name_of_partition>",
                "--pool-member-type=nodeport",
                "--agent=as3",
                "--ipam=true", //Enable IPAM
                ]
          imagePullSecrets:
            - name: f5-docker-images
            - name: bigip-login
    

    cis-deployment-ipam-enabled.yaml

  3. Below is the F5 IPAM Controller Deployment:

    f5-ipam-deployment.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        name: f5-ipam-controller
      name: f5-ipam-controller
      namespace: kube-system
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: f5-ipam-controller
      template:
        metadata:
          labels:
            app: f5-ipam-controller
        spec:
          containers:
          - args:
            - --orchestration=kubernetes
            - --ip-range='{"Dev":"172.16.3.21-172.16.3.30","Test":"172.16.3.31-172.16.3.40", "Production":"172.16.3.41-172.16.3.50",
              "Default":"172.16.3.51-172.16.3.60"}'
            - --log-level=DEBUG
            command:
            - /app/bin/f5-ipam-controller
            image: f5devcentral/f5-ipam-controller:0.1.0
            imagePullPolicy: IfNotPresent
            name: f5-ipam-controller
          serviceAccount: ipam-ctlr
          serviceAccountName: ipam-ctlr
    

    f5-ipam-deployment.yaml

    Push this configuration with the following command:

    kubectl create -f f5-ipam-deployment.yaml
    
  4. Below is the F5 IPAM Controller schema:

    f5-ipam-schema.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    apiVersion: apiextensions.k8s.io/v1
    kind: CustomResourceDefinition
    metadata:
      name: ipams.fic.f5.com
    spec:
      group: fic.f5.com
      names:
        kind: IPAM
        listKind: IPAMList
        plural: ipams
        singular: ipam
      scope: Namespaced
      versions:
        - name: v1
          served: true
          storage: true
          schema:
            openAPIV3Schema:
              type: object
              properties:
                spec:
                  type: object
                  properties:
                    hostSpecs:
                      type: array
                      items:
                        type: object
                        properties:
                          host:
                            type: string
                          key:
                            type: string
                          cidr:
                            type: string
                          ipamLabel:
                            type: string
                status:
                  type: object
                  properties:
                    ipStatus:
                      type: array
                      items:
                        type: object
                        properties:
                          host:
                            type: string
                          key:
                            type: string
                          cidr:
                            type: string
                          ip:
                            type: string
                          ipamLabel:
                            type: string
    

    f5-ipam-schema.yaml

    Push this configuration with the following command:

    kubectl create -f f5-ipam-schema.yaml
    

Configuring IPAM

To configure CIS to work with the F5 IPAM controller, the user needs to provide a parameter --ipam=true in the CIS deployment and also provide a parameter ipamLabel in the Kubernetes resource.

Note

ipamLabel can have values as mentioned in the ip-range parameter in the deployment. For example: -  ipamLabel : "Dev"


virtual-server-crd.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
apiVersion: "cis.f5.com/v1"
kind: VirtualServer
metadata:
 name: coffee-virtual-server
 labels:
   f5cr: "true"
spec:
 host: coffee.example.com
 ipamLabel: Dev
 pools:
 - path: /coffee
   service: svc-2
   servicePort: 80

Note

If you provide the parameter --ipam=true in the CIS deployment, then CIS decides if it needs to retrieve an IP Address from the IPAM Controller or not.

  • If a VirtualServer Address is specified in the Kubernetes resource, CIS will not leverage the IPAM Controller for IP address even if a CIDR parameter is specified.
  • If No VirtualServer Address is specified in the Kubernetes resource and ipamLabel parameter is specified, CIS will leverage the IPAM Controller for allocation of IP address.
transport-server-crd.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: cis.f5.com/v1
kind: TransportServer
metadata:
  creationTimestamp: "2021-03-18T14:12:32Z"
  generation: 2
  labels:
    f5cr: "true"
spec:
  ipamLabel: Test
  mode: standard
  pool:
    monitor:
      interval: 20
      timeout: 10
      type: tcp
    service: test-svc
    servicePort: 1344
  snat: auto
  type: tcp
  virtualServerPort: 1344

Updating the Status in Virtual Server CRD

The main aim of IPAM is to provide an IP address corresponding to each hostname provided in the VS CRD.

The user must provide the host and ipamLabel in the hostSpecs section of F5-CR. The F5 IPAM Controller, in turn, reads the hostSpecs of CR, processes it, and updates the IPStatus with each host provided in the hostSpecs with host, IP (which is generated from the range of IP address by FIC), and corresponding ipamLabel.

Below is the example of F5-CR for Virtual Server:

f5-ipam-cr-vs.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
apiVersion: "fic.f5.com/v1"
kind: F5IPAM
metadata:
  name: f5ipam.sample
  namespace: kube-system
spec:
  hostSpecs:
  - host: cafe.example.com
    ipamLabel: Dev
status:
  IPStatus:
  - host: cafe.example.com
    ip: 172.16.3.16
    ipamLabel: Dev

Updating the Status in Transport Server CRD

The user must provide ipamLabel in the hostSpecs section of F5-CR. The F5 IPAM Controller, in turn, reads the hostSpecs of CR, processes it, and updates the IPStatus with each ipamlabel provided in the hostSpecs with IP (which is generated from the range of IP address by FIC), and corresponding ipamLabel and key which is the combination of <namespace>/<ts_crd_name>_ts.

Below is the example of F5-CR for Transport Server:

f5-ipam-cr-ts.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
apiVersion: "fic.f5.com/v1"
kind: F5IPAM
metadata:
  name: f5ipam.sample
  namespace: kube-system
spec:
  hostSpecs:
  - ipamLabel: Production
    key: default/test-cr-ts1_ts
  - ipamLabel: Test
    key: default/test-cr-ts_ts
status:
  IPStatus:
  - ip: 172.16.3.16
    ipamLabel: Production
    key: default/test-cr-ts1_ts
  - ip: 10.192.75.114
    ipamLabel: Test
    key: default/test-cr-ts_ts

Limitations:

  • A single IPAM Controller does not work with multiple CIS deployments.

  • Sometimes IPAM misses allocating an IP for a domain when CIS is restarted. Mitigation: In this case, you can delete the F5-IPAM custom resource from kube-system named ipam.<Partition_Name> and restart both the controller.

    kubectl delete f5ipam ipam.<Partition_Name> -n kube-system
    

Note

To provide feedback on Container Ingress Services or this documentation, you can file a GitHub Issue.