Fluentd Logging

Overview

The Cloud-Native Network Functions (CNFs) Fluentd Pod is an open-source data collector that can be configured to receive logging data from the BIG-IP Controller, Traffic Management Microkernel (TMM), Edge Firewall, Distributed Session State Managment (dSSM) Pods, and BGP updates from the TMM routing container. The Fluentd Pod must bind to a Kubernetes persistence volume in order to create the necessary log file directories.

This document guides you through understanding, configuring and deploying the f5-fluentd logging container.

Fluentd Service

When installing Fluentd, a Service object is created to receive logging data on TCP service port 54321, and forward the data to Fluentd on TCP service port 24224.

Example Fluentd Service:

Name:              f5-toda-fluentd
Namespace:         cnf-gateway
IP:                10.109.102.215
Port:              <unset>  54321/TCP
Endpoints:         10.244.1.75:24224

Log file locations

Fluentd collects logging data in the following log files:

Container Log file
f5-dssm-sentinel /var/log/f5/f5-dssm-sentinel-0/sentinel.log
f5-dssm-db /var/log/f5/f5-dssm-db-0/dssm.log
f5ingress /var/log/f5/helm_release-f5ingress/pod_name/f5ingress.log
f5-tmm /var/log/f5/f5-tmm/pod_name/f5-fsm-tmm.log
f5-tmm-routing /var/log/f5/f5-tmm/pod_name/f5-tmm-routing.log

_images/spk_info.png Note: To modify the TMM logging level, review the tmm_cli section of the Debug Sidecar overview.

Requirements

Prior to installing Fluentd, ensure you have:

Procedures

Installation

Use the following steps to the install the f5-fluentd container.

  1. Change into the directory containing the latest CNFs Software, and obtain the f5-toda-fluentd Helm chart version:

    In this example, the CNF files are in the cnfinstall directory:

    cd cnfinstall
    
    ls -1 tar | grep f5-toda-fluentd
    

    The example output should appear similar to the following:

    f5-toda-fluentd-1.2.0.tgz
    
  2. Create a new Project for the CNFs Pods using the following command syntax:

    oc create ns <project name>
    

    In this example, a new Project named cnf-gateway is created:

    oc create ns cnf-gateway
    
  3. Create a Helm values file named fluentd-values.yaml, and set the image.repository, persistence.storageClass parameters:

    image:
      repository: <registry>
    
    persistence:
      enabled: true
      storageClass: "<name>"
    

    In this example, Helm pulls the f5-fluentd image from registry.com, and the container will bind to the storageClass named managed-nfs-storage:

    image:
      repository: registry.com
    
    persistence:
      enabled: true
      storageClass: "managed-nfs-storage"
    
  4. In fluentd-values.yaml file set the serviceAccount.create parameter:

    Note: The serviceAccount will not be created by default.

    serviceAccount:
      create: false
      name: default
    
  5. Required: Add the following parameters to the values file to collect logging data from each of the enabled CNFs Pods:

    _images/spk_warn.png Important: The Fluentdbit and Fluentd configurations are mandatory for proper log file recovery in the event of pod or container restarts; without this configuration, the users will not able to receover their log files.

    f5ingress_logs:
      enabled: true
      stdout: true
    dssm_logs:
      enabled: true
      stdout: true
    dssm_sentinel_logs:
      enabled: true
      stdout: true
    pccd_logs:
      enabled: true
      stdout: true
    ipsd_logs:
      enabled: true
      stdout: true
    cm_logs:
    ## Enable Cert manager logging
      enabled: true
      stdout: true
    
  6. Add the F5 Toda Fluentd serviceAccount to the Project’s privileged security context constraint (SCC):

    _images/spk_info.png Note: The cnf-cwc serviceAccount name is based on the Helm release name. See Step 7.

    oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
    

    In this example, the f5-fluentd-f5-toda-fluentd serviceAccount is added to the cnf-gateway Project’s privileged SCC:

    oc adm policy add-scc-to-user privileged -n cnf-gateway -z f5-fluentd-f5-toda-fluentd
    
  7. Install the Fluentd Pod using the following command syntax:

    Note: Be certain to save the Fluentd hostname from the command output for the Controller installation.

    helm install <release> tar/<helm-chart>.tgz \
    -f <values>.yaml -n <namespace>
    

    For example:

    helm install f5-fluentd f5-toda-fluentd-1.2.0.tgz \
    -f fluentd-values.yaml -n cnf-gateway
    

    In the example output, the Fluentd hostname is f5-toda-fluentd.cnf-gateway.svc.cluster.local.:

    FluentD hostname: f5-toda-fluentd.cnf-gateway.svc.cluster.local.
    FluentD port: "54321"
    
  8. The Fluent Pod should now be successfully installed:

    oc get pods -n cnf-gateway
    

    In this example, the Fluentd Pod STATUS is Running:

    NAME                              READY   STATUS    
    f5-toda-fluentd-8cf96967b-jxckr   1/1     Running  
    
  9. Fluentd should also be bound to the persistent volume:

    oc get pvc -n cnf-gateway
    

    In this example, the Fluentd Pod PVC displays STATUS as Bound:

    NAME              STATUS   VOLUME                                     STORAGECLASS
    f5-toda-fluentd   Bound    pvc-7d36b530-b718-466c-9b6e-895e8f1079a2   managed-nfs-storage
    

Viewing logs

After installing the BIG-IP Controller and dSSM Pods, you can use the following steps to view the logs in the f5-fluentd container:

  1. Log in to the fluentd container:

    oc exec -it deploy/f5-toda-fluentd -n <project> -- bash
    

    In this example, the container is in the cnf-gateway Project:

    oc exec -it deploy/f5-toda-fluentd -n cnf-gateway -- bash
    
  2. Change to the main logging directory, and list the subdirectories:

    cd /var/log/f5; ls
    

    In this example, logging directories are present for the f5ingress, f5-tmm, f5-dssm-db, and f5-dssm-sentinel Pods:

    f5-dssm-db-0  f5-dssm-db-1  f5-dssm-db-2  f5-dssm-sentinel-0  
    f5-dssm-sentinel-1  f5-dssm-sentinel-2  f5-ingress-f5ingress  f5-tmm  
    
  3. Change into one of the subdirectories, for example f5-dssm-db-0:

    cd f5-dssm-db-0
    
  4. View the logs using the more command:

    more -d dssm.log
    

Next step

Continue to one of the following steps listed by installation precedence:

Feedback

Provide feedback to improve this document by emailing cnfdocs@f5.com.