Cloud Docs Home > F5 Application Services Proxy Index

Set up the ASP ephemeral store - Kubernetes

The Application Services Proxy (ASP) ephemeral store is a distributed, in-memory, secure key-value store. It allows multiple ASP instances to share non-persistent, or ephemeral, data.

Attention

In Kubernetes v1.4, the ASP ephemeral store uses an alpha resource (PetSet). You must have alpha resources enabled on the Kubernetes API server in order to use the ASP with the ephemeral store. See the Kubernetes API overview for more information.

Important

The ASP does not watch the Kubernetes API for changes to ConfigMaps. This means that you must set up the ASP ephemeral store before you deploy the ASP.

Warning

You cannot use the ephemeral store with ASP v1.0.0. If you have ASP v1.0.0 running, remove it and create new resources for ASP v1.1.0.

Delete the ConfigMap, Deployment, and kill all ASP pods:

$ kubectl delete -f f5-asp-k8s-example-configmap.yaml
$ kubectl delete -f f5-asp-k8s-example-daemonset.yaml
$ kubectl delete pods -l name=f5-asp -n kube-system

Tip

OpenShift users: Substitute oc for kubectl in the example commands.

Set up authentication to the ephemeral store

All communications between clients and the ASP ephemeral store use SSL encryption. Complete the tasks in this section to create and secure the certificates required for authentication to the ephemeral store.

  1. Generate certificates

    Tip

    When working from the examples provided below, change “myuser” to the username of the account needing access the ephemeral store data.

  2. Create the Root Certificate Authority for the ephemeral store. This is a self-signed rootCA certificate and key.

    openssl genrsa -out rootCA.key 2048
    openssl req -new -key rootCA.key -out rootCA.csr -subj "/CN=rootCA"
    openssl x509 -req -days 365 -in rootCA.csr -signkey rootCA.key -out rootCA.crt
    
  3. Create certificates for users. The ephemeral store uses these certificates to authenticate with the server.

    Attention

    • The common name (/CN) provided should match a username defined in the “ephemeral store user” parameter.
    • Use the Root Certificate to sign the user certificates (line 3 in the example below).

    openssl genrsa -out myuser.key 2048
    openssl req -new -key myuser.key -out myuser.csr -subj "/CN=myuser"
    openssl x509 -req -days 365 -in myuser.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out myuser.crt
    
  4. Create Secrets for the certificates

    Run the commands shown below to encrypt the certificates as Kubernetes Secrets. This example creates three (3) Secrets.

    • Include the first (ephemeral-store-root) in the ephemeral store StatefulSet/PetSet.
    • Provide the second and third in the ASP Daemonset. These allow ASP instances to access the ephemeral store.

    Note

    The third secret should contain only the root certificate, without the key. Be sure to create this certificate-only secret in the kube-system namespace.

    Listing 8 Kubernetes
    kubectl create secret tls ephemeral-store-root --cert=rootCA.crt --key=rootCA.key
    kubectl create secret tls ephemeral-store-myuser --cert=myuser.crt --key=myuser.key -n kube-system
    kubectl create secret generic ephemeral-store-user-rootca-cert --from-file=rootCA.crt -n kube-system
    

Set up RBAC for the ephemeral store

  1. Create a Service Account and ClusterRole binding to give the ephemeral store access to the API server.

    # for use with the ASP ephemeral store in Kubernetes v1.6 and later
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: f5-asp
      # namespace in which the ASP runs (if not default)
      namespace: <namespace>
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: f5-asp
      # namespace in which the ASP runs (if not default)
      namespace: <namespace>
    subjects:
      - kind: ServiceAccount
        # Name of the ServiceAccount
        name: f5-asp
        # Must be the same namespace defined in the ServiceAccount
        namespace: <namespace>
    roleRef:
      kind: ClusterRole
      # "Cluster-admin" provides super-user access to all resources in the cluster;
      # for more restrictive access, try "admin" or "system:node"
      name: cluster-admin
      apiGroup: rbac.authorization.k8s.io
    

    Download f5-k8s-asp-rbac.yaml

    kubectl apply -f f5-k8s-asp-rbac.yaml [--namespace=<namespace>]
    

Deploy the ephemeral store

The ephemeral store deployment consists of a ConfigMap, two (2) Services, and a StatefulSet/PetSet. [2] You can define all of the required resources in a single YAML file (referred to as a manifest).

Important

  • Each ephemeral store Pod requires 1 CPU and at least 1Gi memory.
  • By default, the ephemeral store PetSet deploys five (5) Pods. Do not deploy the ephemeral store with fewer than five instances or you may experience data loss.
  1. Create a manifest containing the ASP ephemeral store resources.

    # The configurations for the ephemeral store
    # Each "name" value must match the ``/CN``, or Common Name, defined for a
    # user certificate
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: ephemeral-store-users
    data:
      # Define the ephemeral store users
      user: '{ "name" : "myuser", "auth_mode" : "certificate" }'
    ---
    # A headless service used for DNS resolution
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        role: ephemeral-store-info-service
      name: ephemeral-store-info
    spec:
      ports:
        - name: proto
          protocol: TCP
          port: 8087
          targetPort: 8087
      selector:
        name: ephemeral-store
      clusterIP: None
    ---
    # The ephemeral store service that exposes the ephemeral store Pods
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        role: ephemeral-store-service
      name: ephemeral-store
    spec:
      ports:
        - name: proto
          protocol: TCP
          port: 8087
          targetPort: 8087
      selector:
        name: ephemeral-store
    ---
    # Creates a cluster of five (5) pods running the ephemeral store service.
    # For Kubernetes v1.4.x use PetSet:
    #apiVersion: apps/v1alpha1
    #kind: PetSet
    # For v1.6.x and higher use StatefulSet:
    apiVersion: apps/v1beta1
    kind: StatefulSet
    metadata:
      name: ephemeral-store
    spec:
      serviceName: "ephemeral-store-info"
      replicas: 5
      template:
        metadata:
          labels:
            name: ephemeral-store
    # NOTE: Add the annotation below in Kubernetes v1.4
    #     annotations:
    #       pod.alpha.kubernetes.io/initialized: "true"
        spec:
          terminationGracePeriodSeconds: 0
          containers:
          - name: ephemeral-store
            # The ephemeral store image
            image: f5networks/ephemeral-store:1.0.0
            env:
              - name: NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: INFO_SERVICE
                value: "ephemeral-store-info"
              - name: POD_LABEL_SELECTOR
                value: "ephemeral-store"
              - name: EPHEMERAL_STORE_USER
                valueFrom:
                  configMapKeyRef:
                    # name of the ConfigMap containing the ephemeral store configs
                    name: ephemeral-store-users
                    key: user
            ports:
            - containerPort: 8087
            resources:
              requests:
                cpu: "1"
                memory: "1Gi"
            volumeMounts:
             - mountPath: /var/lib/riak
               name: platform-data-dir
             - mountPath: /var/log/riak
               name: platform-log-dir
             - mountPath: /riak-bootstrap/ca-certs
               name: ca-certs
            livenessProbe:
              exec:
                command:
                - riak-admin
                - test
              initialDelaySeconds: 15
              periodSeconds: 30
            readinessProbe:
              exec:
                command:
                - /riak-bootstrap/readiness-probe.sh
              initialDelaySeconds: 15
              periodSeconds: 30
          volumes:
            - name: platform-data-dir
              emptyDir: {}
            - name: platform-log-dir
              emptyDir: {}
            - name: ca-certs
              secret:
                # name of the Secret containing both the root certificate and key
                secretName: ephemeral-store-root
    

    ephemeral-store-k8s-example.yaml

  2. Upload the manifest to the Kubernetes API server.

    kubectl create -f f5-ephemeral-store-k8s-example.yaml
    configmap "ephemeral-store-users" created
    service "ephemeral-store-info" created
    service "ephemeral-store" created
    petset "ephemeral-store" created
    
  3. Verify creation of the ephemeral store cluster. You should see five (5) Pods.

    Important

    Due to the ephemeral store size requirements, the initial launch of these Pods can take a few minutes. If a Pod stays in “pending” status for an extended period of time, you may have run out of CPU.

    Run kubectl describe pods <pod_name> to debug.

    kubectl get pods -o wide
    NAME                       READY     STATUS              RESTARTS   AGE       IP          NODE
    ephemeral-store-0          1/1       Running             0          1m        10.2.55.3   172.16.1.183
    ephemeral-store-1          1/1       Running             0          1m        10.2.76.3   172.16.1.184
    ephemeral-store-2          1/1       Running             0          1m        10.2.76.4   172.16.1.184
    ephemeral-store-3          1/1       Running             0          1m        10.2.55.5   172.16.1.183
    ephemeral-store-4          1/1       Running             0          1m        10.2.76.5   172.16.1.184
    

Next Steps

Once you’ve set up the ephemeral store, install and deploy the ASP.

Learn More

See Set up the ASP ephemeral store - Kubernetes and Health Monitors.

Footnotes

[1]See Set Pod CPU & Memory Limit .
[2]PetSets changed to StatefulSets in Kubernetes v1.5.