Cloud-Init and F5 BIG-IP Virtual Edition

Version notice:

F5® BIG-IP® Virtual Edition (VE) 13.0.0 and later only, unless otherwise indicated.

Cloud-Init is the industry standard start-up agent installed on virtual machines to facilitate cloud deployments. Beginning with BIG-IP VE 13.0.0, Cloud-Init is automatically installed on some platforms (see the following list of supported hypervisors). You can speed up the initialization of your BIG-IP instance by passing user-data to perform tasks like onboarding and configuring BIG-IP. For example, to pass user-data you can use a bash startup script or cloud-init’s built-in yaml-based configuration file, a cloud-config file.

Hypervisors with Cloud-Init support in BIG-IP VE

Cloud-Init is automatically installed when you use BIG-IP VE on the following hypervisors:

• Citrix XenServer
• Linux KVM
• Linux Xen Project
• Microsoft Hyper-V
• VMware ESXi
• Amazon Web Services EC2
• Microsoft Azure (supports BIG-IP VE 15.1 and later)
• Alibaba (BIG-IP VE 14.1 and later)
• Google Cloud Platform (supports BIG-IP VE 15.1 and later)

Deploy with Cloud-Init

To pass the user configuration to the virtual machine, you can do one of the following:

Option One: Copy and paste the data using the cloud provider’s web interface, for example in AWS:

AWS’s Configure Instance Details - Advanced Details section

Option Two: Pass as a parameter (string or filename) in the provider’s API to launch the instance:

• OpenStack API user-data, or using the CLI user-data file:

  openstack server create --image my_image --flavor 1 --user-data myuserdata.yaml VM_INSTANCE
  

• AWS API UserData, or using the CLI user-data:

  aws ec2 run-instances --image-id ami-abc1234 --count 1 --instance-type m4.large --key-name keypair --user-data file://myuserdata.yaml --subnet-id subnet-abcd1234 --security-group-ids sg-abcd1234
  

• Azure API customData, or using the CLI custom_data:

  az vm create -g MyResourceGroup -n MyVm --image myimage --custom-data myuserdata.yaml
  

• GCP API metadata (key = startup-script), or the CLI metadata-from-file startup-script=:

  gcloud compute instances create example-instance --metadata-from-file startup-script=myuserdata.yaml
  

Option Three: For providers that lack a dedicated user-data field included in their API, place the user-data in an ISO (OpenStack Config Drive v2 format) that you attach to the instance at launch. For example:

1. To create an ISO, you can use tools like mkisofs:

   mkisofs -R -V config-2 -o <path-to-iso> <path-to-iso-dir>
   

   Or, consult this TMOS Config Drive Builder Python script.

2. To deploy with config drive:

   • KVM

     • virt-install:

       virt-install -n bigip1 -r 4096  -w model=virtio,network=default --disk ~/BIGIP-14.1.0.6-0.0.9.qcow2,bus=virtio,size=8 --import --wait 0 --os-variant rhel7 --vcpus=2 --noautoconsole --disk path=~/configdrives/configdrive.iso,device=cdrom
       

   • VMware

     See the Advanced configuration using Clouid-Init topic.

Cloud-Init data sources supported in BIG-IP VE

This version of BIG-IP VE supports the following data sources.

• The EC2 data source in AWS and OpenStack.
• Config Drive V2 in OpenStack and all non-cloud hypervisors.
• The F5Azure data source (replaces the built-in data source) for use with BIG-IP VE in Microsoft Azure. This data source parses the ovf-env.xml file to get all metadata, configuration, and user data.

Cloud-Init modules supported in BIG-IP VE

BIG-IP VE supports a limited subset of Cloud-Init modules.

• Final Message - Configures the final message that cloud-init writes. The message is specified as a jinja template with the following variables set: version, timestamp, datasource, and uptime.
• Runcmd - Run arbitrary commands at a rc.local-like level with output to the console. See the following example of cloud-config using this module.
• Scripts Per Boot - Runs any scripts in the scripts/per-boot directory on the datasource every time the system boots.
• Scripts Per Instance - Runs any scripts in the scripts/per-instance directory on the datasource when a new instance is first booted.
• Scripts Per Once - Runs any scripts in the scripts/per-once directory on the datasource only once. Scripts will run in alphabetical order.
• Scripts User - Runs all user scripts. User scripts are not specified in the scripts directory in the datasource, but rather are present in the scripts dir in the instance configuration.
• Write Files - Write out arbitrary content (plain text or binary) to files, optionally setting permissions. See the following example of cloud-config using this module.
• Set Password (for use with BIG-IP VE 15.1 and later) - Changes the built-in TMOS admin and root passwords.
• TMOS Declared (for use with BIG-IP VE 15.1 and later) - Custom F5 module that facilitates leveraging F5 Automation Toolchain (including, F5 Declarative Onboarding and F5 Application Services Extension).

Cloud-Init examples

The following examples demonstrate built-in cloud-init modules and custom cloud-init modules for deploying BIG-IP.

Cloud-config using Write Files and Runcmd modules example

Consult the following cloud-config using the Write Files and Runcmd modules:

#cloud-config
write_files:
  - path: /config/custom-config.sh
    permissions: 0755
    owner: root:root
    content: |
      #!/bin/bash

      echo "Hello World" >> /var/tmp/cloud-init-output

      # Wait for MCPD to be up before running tmsh commands
      source /usr/lib/bigstart/bigip-ready-functions
      wait_bigip_ready

      # Begin BIG-IP configuration
      tmsh modify sys global-settings gui-setup disabled
      tmsh modify sys global-settings gui-security-banner-text "Configured via Cloud-Init!"

      tmsh save /sys config
runcmd:
  # NOTE: Commands must be non-blocking so send long running commands (polling/waiting for mcpd) to the background
  - /config/custom-config.sh &

Important

Because cloud-init executes early in the boot process, if you expect your Cloud-Init script to take more than 10 seconds to execute, then background or fork to a new process so your script is non-blocking and BIG-IP VE can continue booting.

Set Password module example

The cloud-init configuration template for BIG-IP VE 15.1 and later supports the standard cloud-init set_password module. To build TMOS admin and root passwords, use the following example of the cloud-init cloud-config declarations:

#cloud-config
chpasswd:
  list: |
    root:f5str0ngPa$$word
    admin:f5str0ngPa$$word
  expire: False

Important

Some provider’s metadata are insecure and user-data content is unencrypted and queryable throughout the instance’s lifetime. Therefore, never send sensitive data such as passwords via this file. Consult your provider’s best practices (for example, AWS).

TMOS Declared module example

Use the custom Cloud-Init TMOS Declared module with BIG-IP VE 15.1 and later to simplify deploying BIG-IP through cloud-init and Automation Toolchain, including F5 Declarative Onboarding and F5 Application Services Extension.

This custom Cloud-Init, TMOS Declared module requires the following:

• Management interface provisioning completed using either the DHCPv4 or DHCPv6 default method.
• All other on-board configurations handled using the f5-declarative-onboarding and f5-appsvcs-extension declarations.

Module Attribute	Default Value	Rquired	Description
enabled	false	Yes	Activates the module.
icontrollx_trusted_sources	true	No	Indicating to only install iControl LX RPMs, which are signed by TMOS trusted keys.
icontrollx_package_urls	none	Yes	List of URLs to download and install iControl LX extension packages before on-boarding.
do_declaration	none	No	YAML-formatted f5-declarative-onboarding declaration, augmenting or overwriting the declaration created by resource discovery.
as3_declaration	none	No	The F5 BIG-IP Application Services 3 Extension (AS3) to declare, if enabled.
phone_home_url	url	No	Reachable URL used to report completion of this module-onboarding.
phone_home_url_verify_tls	true	No	If the phone_home_url uses TLS, verify the host certificate.
phone_home_cli	cli command	No	CLI command to run when this module-onboarding completes successfully.

Tip

You can use the Set Password cloud-init module to change the default admin and root passwords, rather than the f5-declarative-onboarding declaration to change user passwords; however, both ways will work.

Userdata usage

#cloud-config
tmos_declared:
tmos_declared:
  enabled: true
  icontrollx_trusted_sources: false
  icontrollx_package_urls:
    - "https://github.com/F5Networks/f5-declarative-onboarding/raw/master/dist/f5-declarative-onboarding-1.3.0-4.noarch.rpm"
    - "https://github.com/F5Networks/f5-appsvcs-extension/raw/master/dist/latest/f5-appsvcs-3.10.0-5.noarch.rpm"
    - "https://github.com/F5Networks/f5-telemetry-streaming/raw/master/dist/f5-telemetry-1.2.0-1.noarch.rpm"
  do_declaration:
    schemaVersion: 1.0.0
    class: Device
    async: true
    label: Cloudinit Onboarding
    Common:
      class: Tenant
      provisioningLevels:
        class: Provision
        ltm: nominal
        asm: nominal
      poolLicense:
        class: License
        licenseType: licensePool
        bigIqHost: licensor.example.openstack.com
        bigIqUsername: admin
        bigIqPassword: admin
        licensePool: BIGIPVEREGKEYS
        reachable: true
        bigIpUsername: admin
        bigIpPassword: admin
      dnsServers:
        class: DNS
        nameServers:
          - 8.8.8.8
        search:
          - example.openstack.com
      ntpServers:
        class: NTP
        servers:
          - 0.pool.ntp.org
          - 1.pool.ntp.org
          - 2.pool.ntp.org
      internal:
        class: VLAN
        mtu: 1450
        interfaces:
          - name: 1.2
            tagged: false
      internal-self:
        class: SelfIp
        address: 192.168.40.51/24
        vlan: internal
        allowService: default
        trafficGroup: traffic-group-local-only
      external:
        class: VLAN
        mtu: 1450
        interfaces:
          - name: 1.3
            tagged: false
      external-self:
        class: SelfIp
        address: 192.168.80.56/24
        vlan: external
        allowService: none
        trafficGroup: traffic-group-local-only
      default:
        class: Route
        gw: 192.168.80.1
        network: default
        mtu: 1500
      dbvars:
        class: DbVariables
        ui.advisory.enabled: true
        ui.advisory.color: orange
        ui.advisory.text: This device is under centralized management.
  as3_declaration:
    class: ADC
    schemaVersion: 3.0.0
    label: ASM_VS1
    remark: ASM_VS1
    Sample_app_sec_01:
      class: Tenant
      HTTP_Service:
        class: Application
        template: http
        serviceMain:
          class: Service_HTTP
          virtualAddresses:
            - 192.168.80.51
          snat: auto
          pool: Pool1
          policyWAF:
            use: WAFPolicy
        Pool1:
          class: Pool
          monitors:
            - http
          members:
            - servicePort: 8001
              serverAddresses:
                - 10.10.10.143
            - servicePort: 8002
              serverAddresses:
                - 10.10.10.144
        WAFPolicy:
          class: WAF_Policy
          url: "https://raw.githubusercontent.com/f5devcentral/f5-asm-policy-template-v13/master/owasp_ready_template/owasp-no-autotune.xml"
          ignoreChanges: true
  post_onboard_enabled: true
  post_onboard_commands:
    - "echo 'curl -s http://monitors.internal.local/rebooted' >> /config/startup"
  phone_home_url: "https://webhook.site/5f8cd8a7-b051-4648-9296-8f6afad34c93"
  phone_home_cli: "curl -i -X POST -H 'X-Auth-Token: gAAAAABc5UscwS1py5XfC3yPcyN8KcgD7hYtEZ2-xHw95o4YIh0j5IDjAu9qId3JgMOp9hnHwP42mYA7oPPP0yl-OQXvCaCS3OezKlO7MsS-ZCTJzuS3sSysIMHTA78fGsXbMgCQZCi5G-evLG9xUNrYp5d3blhMnpHR0dlHPz6VMacNkPhyrQI' -H 'Content-Type: application/json' -H 'Accept: application/json' http://192.168.0.121:8004/v1/d3779c949b57403bb7f703016e91a425/stacks/demo_waf/3dd6ce45-bb8c-400d-a48c-87ac9e46e60e/resources/wait_handle/signal"

Tip

Example declarations for Tool Chain APIs are in json format (for example, DO and F5 BIG-IP Application Services 3 Extension (AS3)). To convert these examples to yaml format for use in cloud-config, use a conversion tool.

The application listening at the phone_home_url must accept a POST request. The POST body is a JSON object with the following format:

{
   "id": "a67d1edb-0a4a-4101-afd1-2fbf04713cfa",
   "version": "14.1.0.1-0.0.7.0",
   "product": "BIGIP",
   "hostname": "waf1primary.local",
   "management": "192.168.245.119/24",
   "installed_extensions": ["f5-service-discovery", "f5-declarative-onboarding", "f5-appsvcs"],
   "as3_enabled": true,
   "do_enabled": true,
   "status": "SUCCESS"
}

If the module runs successfully and the provisioning can be synchronized, then the phone_home_cli is called. The phone_home_cli command execution enables OpenStack Heat and AWS CFT type to use wait condition resources with their auto-generated curl CLI notifications.

Troubleshooting Cloud-Init for BIG-IP

Use this section to consult log files for troubleshooting purposes and to verify that Cloud-Init is initiated correctly on the BIG-IP VE image.

Troubleshoot Cloud-Init and the TMOS Declared module

VE administrators can check the files with the following naming conventions:

Cloud-Init:

• /var/log/boot.log
• /var/log/cloud*.log

Custom TMOS Declared module:

• Logs: /var/log/f5-cloud*.log
• Json declarations created by custom TMOS Declared module (if provided):
  • /var/lib/cloud/f5-declarative-onboarding
  • /var/lib/cloud/f5-appsvcs-extension

Determine if Cloud-Init is on BIG-IP VE

If you experience trouble with Cloud-Init, confirm that it was properly installed and initiated:

1. Use SSH to connect to your BIG-IP VE instance as a privileged user.

2. Type, bash to enter the bash shell and search for:

   > egrep 'running|finished' /var/log/cloud-init-output.log
   

3. There are the following entries in the log. Find the most recent entries that contain cloud-init; for example:

   # egrep 'running|finished' /var/log/cloud-init-output.log
   Cloud-init v. 0.7.5 running 'init-local' at Tue, 27 Aug 2019 23:43:00 +0000. Up 89.65 seconds.
   Cloud-init v. 0.7.5 running 'init' at Tue, 27 Aug 2019 23:43:00 +0000. Up 89.88 seconds.
   Cloud-init v. 0.7.5 running 'modules:config' at Tue, 27 Aug 2019 23:43:01 +0000. Up 90.55 seconds.
   Cloud-init v. 0.7.5 running 'modules:final' at Tue, 27 Aug 2019 23:43:01 +0000. Up 90.85 seconds.
   Cloud-init v. 0.7.5 finished at Tue, 27 Aug 2019 23:43:01 +0000. Datasource DataSourceEc2.  Up 90.96 seconds
   

   These entries indicate that when BIG-IP VE booted, cloud-init successfully triggered.

Rerun cloud-init without rebooting

rm -f /var/log/cloud-init-output.log \
&& rm -Rf /var/lib/cloud/* \
&& cloud-init -d init \
&& cloud-init -d modules --mode final

Disable Cloud-Init on BIG-IP VE

Cloud-init can run once or at every boot. To stop Cloud-Init from automatically running, do the following:

1. Use SSH to connect to your BIG-IP VE instance.

2. Ensure you are at the tmsh prompt.

3. Run this command to disable the Cloud-Init database setting.

   modify sys db service.cloudinit value disable
   

4. Run this command to reboot BIG-IP VE immediately.

   reboot
   

When BIG-IP VE restarts, Cloud-Init is no longer running.

If you decide to re-enable Cloud-Init later, run this command and reboot:

modify sys db service.cloudinit value enable