Azure

In this section, you will see the complete steps for implementing Cloud Failover Extension in Microsoft Azure. You can also go straight to the Example Azure Declaration.

Azure CFE Prerequisites

These are the basic prerequisites for setting up CFE in Microsoft Azure.

  • 2 BIG-IP systems in Active/Standby configuration. You can find an example ARM template here. Any configuration tool can be used to provision the resources.

  • Virtual addresses created in a floating traffic group and matching addresses (secondary) on the IP configurations of the instance NICs serving application traffic.

    Tip

    Use Static allocation for each IP configuration that will serve application traffic. Using Dynamic allocation is discouraged for production deployments.

  • Access to Azure’s Instance Metadata Service, which is a REST Endpoint accessible to all IaaS VMs created with the Azure Resource Manager. The endpoint is available at a well-known non-routable IP address (169.254.169.254) that can only be accessed from within the VM. See the instructions below to Set up access to Azure’s Instance Metadata Service.

  • Enable “enableIPForwarding” on the NICs if enabling routing or avoiding SNAT. See https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-network-interface#enable-or-disable-ip-forwarding

Note

CFE makes calls to the Azure APIs in order to failover cloud resource objects such as private IP addresses and route tables. These calls may vary significantly in response time. See the Performance and Sizing section for example times.


Complete these tasks to deploy Cloud Failover Extension in Microsoft Azure. Before getting started, we recommend you review the Known Issues and Frequently Asked Questions (FAQ).

Task Summary
Step Task

Download the RPM file

Upload and install the Cloud Failover Extension file on each BIG-IP

Create and assign a Managed Service Identity (MSI)

Tag your Azure Network Infrastructure Objects

Set up access to Azure’s Instance Metadata Service
Modify and POST the Example Azure Declaration
Update or Revert Cloud Failover Extension

Azure Failover Event Diagram

The following diagram shows a failover event with CFE implemented in Microsoft Azure with an HA pair in an Active/Standby configuration.

In the diagram, the IP configuration has a secondary private address that matches a virtual address in a traffic group owned by the active BIG-IP. In the event of a failover, the IP configuration is deleted and recreated on that device’s network interface. Simultaneously, the user-defined routes are updated with a next hop attribute the corresponds to the self IP address of the active BIG-IP.

../_images/azure-failover-3nic-multiple-vs-animated.gif

Note

Management NICs/Subnets are not shown in this diagram.

Example Azure Declaration

This example declaration shows the minimum information needed to update the cloud resources in Azure. See the Quickstart section for steps on how to post this declaration. See the Example Declarations section for more examples.

Example Azure Declaration with Single Routing Table
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
   "class":"Cloud_Failover",
   "environment":"azure",
   "controls":{
      "class":"Controls",
      "logLevel":"silly"
   },
   "externalStorage":{
      "scopingTags":{
         "f5_cloud_failover_label":"mydeployment"
      }
   },
   "failoverAddresses":{
      "enabled":true,
      "scopingTags":{
         "f5_cloud_failover_label":"mydeployment"
      }
   },
   "failoverRoutes":{
      "enabled":true,
      "routeGroupDefinitions":[
         {
            "scopingName":"myroutetable-1",
            "scopingAddressRanges":[
               {
                  "range":"0.0.0.0/0"
               }
            ],
            "defaultNextHopAddresses":{
               "discoveryType":"static",
               "items":[
                  "10.0.13.11",
                  "10.0.23.11"
               ]
            }
         }
      ]
   }
 }
 

azure.json


Create and assign a Managed Service Identity (MSI)

In order to successfully implement CFE in Azure, you need a system-assigned or user-managed identity with sufficient access. Your Managed Service Identity (MSI) should be limited to the resource groups that contain the BIG-IP instances, VNET, route tables, etc. that will be updated. Read more about managed identities here. To create and assign a Managed Service Identity (MSI) you must have a role of User Access Administrator or Contributor access. The following example shows a system-assigned MSI.

  1. Enable MSI for each VM: go to Virtual Machine > Identity > System assigned and set the status to On.

    For example:

    ../_images/AzureMSIVMIdentity.png

  2. Assign permissions to each MSI: go to Resource Group > Access control (IAM) > Role assignments > Add, make the changes listed below, and then add the MSI.

    • Role: Contributor
    • Assign access to: System assigned managed identity > Virtual Machine

    For example:

    ../_images/AzureMSIAssignedToResourceGroup.png

Note

Certain resources may be deployed in a separate subscription, add role assignments for each subscription where resources are located.

RBAC Role Definition

Below is an example Azure role definition with permissions required by CFE.

  • Microsoft.Authorization/*/read
  • Microsoft.Compute/locations/*/read
  • Microsoft.Compute/virtualMachines/*/read
  • Microsoft.Network/networkInterfaces/read
  • Microsoft.Network/networkInterfaces/write
  • Microsoft.Network/*/join/action
  • Microsoft.Network/routeTables/*/read
  • Microsoft.Network/routeTables/*/write
  • Microsoft.Resources/subscriptions/resourceGroups/read
  • Microsoft.Storage/storageAccounts/read
  • Microsoft.Storage/storageAccounts/listKeys/action

Important

  • This example provides the minimum permissions required and serves as an illustration. You are responsible for following the provider’s IAM best practices.
  • Certain resources such as the virtual network are commonly deployed in a separate resource group, ensure the correct scopes are applied to all applicable resource groups.
  • Certain resources such as route tables may be deployed in a separate subscription, ensure the assignable scopes applies to all relevant subscriptions.

Tag your Azure Network Infrastructure Objects

Tag your infrastructure with the the names and values that you will send in your CFE declaration.

Important

You must tag the following resources. Even if you only have routes to update during failover (for example, there are no NIC IP configuration objects to re-map) you still have to tag the NICs on the Virtual Machines associated with the IPs in your CFE declaration.

Tag the Storage Account in Azure

Add a storage account to your resource group, and tag with a name/value pair that corresponds to the name/value pair in the externalStorage.scopingTags section of the CFE declaration.

Warning

Ensure the required storage accounts do not have public access.

Tag the Network Interfaces in Azure

  1. Within Azure, go to NIC > Tags. Create two sets of tags for Network Interfaces:

    • Deployment scoping tag: a key-value pair that will correspond to the key-value pair in the failoverAddresses.scopingTags section of the CFE declaration.

      Note

      If you use our declaration example, the key-value tag would be: "f5_cloud_failover_label":"mydeployment"

    • NIC mapping tag: a key-value pair with the reserved key named f5_cloud_failover_nic_map and a user-provided value that can be anything. For example "f5_cloud_failover_nic_map":"external".

      Important

      The same tag (matching key:value) must be placed on corresponding NIC on the peer BIG-IP. For example, each BIG-IP would have their external NIC tagged with "f5_cloud_failover_nic_map":"external" and their internal NIC tagged with "f5_cloud_failover_nic_map":"internal".

Example:

../_images/AzureNICTags.png

Tag the User-Defined routes in Azure

In CFE version 1.5.0, the parameter routeGroupDefinitions was added. It allows more granular route operations and you are not required to tag the routes. Provide the name of the route you want to manage with scopingName. See Failover Routes for more information.

"failoverRoutes":{
    "enabled":true,
    "routeGroupDefinitions":[
        {
          "scopingName":"myroutetable-1",
          "scopingAddressRanges":[
              {
                "range":"0.0.0.0/0"
              }
          ],
          "defaultNextHopAddresses":{
              "discoveryType":"static",
              "items":[
                "10.0.13.11",
                "10.0.13.12"
              ]
          }
        }
    ]
}


To enable route failover in versions earlier than 1.5.0, tag the route tables containing the routes that you want to manage:

  1. In Azure, create a key-value pair that will correspond to the key-value pair in the failoverAddresses.scopingTags section of the CFE declaration.

    Note

    If you use our declaration example, the key-value tag would be "f5_cloud_failover_label":"mydeployment"

  2. In the case where BIG-IP has multiple NICs, CFE needs to know what interfaces (by using the Self-IPs associated with those NICs) it needs to re-map the routes to. You can either define the nextHopAddresses using an additional tag on the route table, or you can provide them statically in the cloud failover configuration.

    • If you use discoveryType routeTag, you will need to add another tag to the route table in your cloud environment with the reserved key f5_self_ips. For example, "f5_self_ips":"10.0.13.11,10.0.13.12".

    "failoverRoutes": {
      "enabled": true,
      "scopingTags": {
        "f5_cloud_failover_label": "mydeployment"
      },
      "scopingAddressRanges": [
        {
          "range": "0.0.0.0/0",
          "nextHopAddresses": {
              "discoveryType":"routeTag"
          }
        }
      ]
    }
    
    • If you use discoveryType static, you can provide the Self-IPs in the items area of the CFE configuration. See Failover Routes for more information.

  1. Within Azure, go to Basic UDR > Tags to create a deployment scoping tag. The name and value can be anything; the example below uses f5_cloud_failover_label:mydeployment.

    ../_images/AzureUDR.png


Set up access to Azure’s Instance Metadata Service

Azure’s Instance Metadata Service is a REST Endpoint accessible to all IaaS VMs created via the Azure Resource Manager. The endpoint is available at a well-known non-routable IP address (169.254.169.254) that can be accessed only from within the VM.

Important

Certain BIG-IP versions and/or topologies may use DHCP to create the management routes (for example: dhclient_route1), if that is the case the below steps are not required.

To configure the route on BIG-IP to talk to Azure’s Instance Metadata Services, use either of the commands below. Note that in this example, 192.0.2.1 is the management subnet’s default gateway.

Using TMSH

tmsh modify sys db config.allow.rfc3927 value enable
tmsh create sys management-route metadata-route network 169.254.169.254/32 gateway 192.0.2.1
tmsh save sys config

Using Declarative Onboarding

{
  "managementRoute": {
    "class": "ManagementRoute",
    "gw": "192.0.2.1",
    "network": "169.254.169.254",
    "mtu": 1500
  },
  "dbVars": {
    "class": "DbVariables",
    "config.allow.rfc3927": "enable"
  }
}

Example Virtual Service Declaration

See below for example Virtual Services created with AS3 in Azure Failover Event Diagram above:

Example AS3 Declaration
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
{
    "remark":"AS3_Azure",
    "schemaVersion":"3.0.0",
    "label":"AS3_Azure",
    "class":"ADC",
    "Tenant_Shared_Services":{
       "class":"Tenant",
       "Shared_Services":{
          "class":"Application",
          "template":"generic",
          "vs_wildcard_forwarding":{
             "class":"Service_Forwarding",
             "remark":"Outbound Wildcard Forwarding Virtual Server",
             "virtualAddresses":[
                [
                   "0.0.0.0",
                   "0.0.0.0/0"
                ]
             ],
             "virtualPort":"0",
             "forwardingType":"ip",
             "layer4":"any",
             "snat":"auto",
             "allowVlans":[
                {
                   "bigip":"/Common/internal"
                }
             ]
          }
       }
    },
    "Tenant_1":{
       "class":"Tenant",
       "Service_1":{
          "class":"Application",
          "template":"https",
          "serviceMain":{
             "class":"Service_HTTPS",
             "remark":"Service 1 on Private IP",
             "virtualAddresses":[
                "10.0.12.101"
             ],
             "snat":"auto",
             "serverTLS":{
                "bigip":"/Common/clientssl"
             },
             "redirect80":false,
             "pool":"Service_1_Pool"
          },
          "Service_1_Pool":{
             "class":"Pool",
             "remark":"Service 1's Pool",
             "members":[
                {
                   "servicePort":80,
                   "serverAddresses":[
                      "10.0.14.101",
                      "10.0.14.102",
                      "10.0.14.103"
                   ]
                }
             ],
             "monitors":[
                "http"
             ]
          }
       }
    },
    "Tenant_2":{
       "class":"Tenant",
       "Service_2":{
          "class":"Application",
          "template":"https",
          "serviceMain":{
             "class":"Service_HTTPS",
             "remark":"Service 2 on Private IP",
             "virtualAddresses":[
                "10.0.12.102"
             ],
             "snat":"auto",
             "serverTLS":{
                "bigip":"/Common/clientssl"
             },
             "redirect80":false,
             "pool":"Service_2_Pool"
          },
          "Service_2_Pool":{
             "class":"Pool",
             "remark":"Service 2's Pool",
             "members":[
                {
                   "servicePort":80,
                   "serverAddresses":[
                      "10.0.14.201",
                      "10.0.14.202",
                      "10.0.14.203"
                   ]
                }
             ],
             "monitors":[
                "http"
             ]
          }
       }
    }
 }

azure-as3.json


Note

To provide feedback on Cloud Failover Extension or this documentation, you can file a GitHub Issue.