Amazon Web Services: High Availability BIG-IP VE

If a BIG-IP VE becomes unavailable for any reason, it can fail over to another BIG-IP VE. In the AWS environment, if the active BIG-IP VE suddenly goes offline, BIG-IP VE drops the active connections. The other BIG-IP VE processes new connections when it becomes active.

The following illustration shows an example of two BIG-IP VE instances in an Amazon VPC. The two BIG-IP VEs are members of a BIG-IP device group, which means that the BIG-IPs trust each other, they synchronize their configurations, and they can fail over to one another.

Each BIG-IP VE has the default floating traffic group, traffic-group-1, that contains a floating virtual IP address. Application traffic is going to the virtual IP address on BIG-IP A. If BIG-IP A goes offline, the virtual IP address becomes active on BIG-IP B and the traffic redirects to it.

../_images/ha1.png

Complete the tasks in this guide to create this deployment.

Alternately, you can use CloudFormation templates to create this deployment. For more information about CloudFormation templates provided by F5, go to https://github.com/F5Networks.

High availability configuration overview

This illustration shows the additional network objects you must create for a typical BIG-IP VE high availability (HA) configuration in AWS.

In this configuration, the BIG-IP VEs continually communicate their availability status to one another through the HA VLAN and the associated static self IP address on each BIG-IP VE.

../_images/ha2.png

To create this configuration, in AWS, you create an HA subnet with primary private IP addresses and network interfaces. Then in BIG-IP VE, you create corresponding objects, represented by the shaded boxes in the diagram.

Task List: Create two BIG-IP VE instances

Follow the steps in the Amazon Web Services: Multi-NIC Configuration guide to create and deploy two BIG-IP VE instances (BIG-IP A and BIG-IP B) in an Amazon VPC with multiple subnets using these HA networking objects.

Note: Both BIG-IP VE instances must be in the same availability zone. For HA across availability zones, see the CFTs on https://github.com/F5Networks.

Task List: Create HA interfaces for both BIG-IP instances (A and B)

Once deployed, these instances will require you to add the following AWS resources for HA:

Step Task Description
1 Create an IAM role to enable communication between BIG-IP VE and AWS and apply the role to both instances (A and B).

An IAM user or role with sufficient policy permissions must exist in AWS. Assign the role to both BIG-IP VE instances, or enter the user’s keys.

To use an IAM role for communication, see this topic. IAM roles work in BIG-IP VE 13.0.0 and later.

To use a IAM user for communication, see this topic.

2 Create an Elastic IP address (EIP) for each BIG-IP Self-IP on the external interface. Once a default route is configured on the external subnet, BIG-IP will use these EIPs (publically routable addresses) to communicate with the AWS API.
3 Create a subnet for HA communication.

The BIG-IPs will use a separate subnet for HA communication.

  • HA Subnet: 10.0.3.0/24
4 Create a network interface (NIC) on the HA subnet for each BIG-IP.

These two NICs are used for HA communication.

  • Interface: eth3
5 Reboot both BIG-IP instances, so they can discover the additional HA NIC. You can use the BIG-IP CLI or the AWS UI.

Task List: Configure high availability on BIG-IP VE

To set up high availability (HA), create these resources. This is a specific example, which you can use to test an HA configuration.

Step Task Description
1 Create VLANs for HA communication.

On each BIG-IP VE, create a VLAN that corresponds to the HA subnet.

  • VLAN: HA
2 Create static self IP addresses for the HA VLANs.

On each BIG-IP VE, create a static self IP address used for failover communication. These IP addresses must match the private IP addresses assigned to the HA subnet in AWS.

  • Self IP on BIG-IP A: 10.0.3.96
  • Self IP on BIG-IP B: 10.0.3.185
3 Establish device trust.

The BIG-IP VEs must establish trust by exchanging certificates. Use management IP addresses to do this.

  • Management IP on BIG-IP A: 10.0.0.200
  • Management IP on BIG-IP B: 10.0.0.201
4 Specify config sync and failover addresses.

These are the static self IP addresses that you want the BIG-IP VEs to use for config sync and failover operations to one another.

Config sync static self IP for internal VLAN:

  • BIG-IP A: 10.0.2.200
  • BIG-IP B: 10.0.2.201

Static self IP for the HA VLAN:

  • BIG-IP A: 10.0.3.96
  • BIG-IP B: 10.0.3.185
5 Create a Sync-Failover device group.

BIG-IP VEs in a Sync-Failover device group can sync their configurations and fail over to one another.

  • bigip_ve_dg
6 Synchronize the BIG-IP configuration. Log into BIG-IP A and sync its configuration to BIG-IP B.

Create a subnet for HA communication

Each BIG-IP VE instance uses three VPC subnets, for management, external, and internal traffic. Note the availability zone for these subnets (for example, us-west-2a).

Now, in the same availability zone, create a subnet for high availability (HA) communication between the two instances. This subnet corresponds to the BIG-IP VLAN named HA that you will create later on each BIG-IP VE.

  1. In the AWS Management Console, from the Services menu at the top of the screen, select VPC.
  2. In the Navigation pane, under Virtual Private Cloud, select Subnets.
  3. Click Create Subnet.
  4. In the Name tag field, type HA.
  5. In the VPC field, select the VPC.
  6. In the Availability Zone field, select the zone where the other subnets reside.
  7. In the CIDR block field, type 10.0.3.0/24.
  8. Click Yes, Create.

Your VPC should now have four subnets:

  • management: 10.0.0.0
  • external: 10.0.1.0
  • internal: 10.0.2.0
  • HA: 10.0.3.0

Create HA network interfaces

Each of your BIG-IP VE instances should have three network interfaces, one per subnet (management, external, and internal). Now create another network interface for each instance and associate it with the HA subnet.

  1. In the AWS Management Console, from the Services menu at the top of the screen, select EC2.

  2. In the Navigation pane, under NETWORK & SECURITY, select Network Interfaces.

  3. Click Create Network Interface and populate the appropriate fields.

    Field Value
    Description HA-A
    Subnet 10.0.3.0/24
    Private IP 10.0.3.96
    Security groups InternalTraffic

    Note: You do not need to create a separate security group for the HA network interfaces.

  4. Click Yes, Create.

    AWS adds the network interface to the list.

  5. Update the name in the list to HA-A.

  6. Right-click the new network interface and select Attach.

  7. From the Instance ID list, select the instance for BIG-IP A and click Attach.

  8. Repeat this task for BIG-IP B, using these values and attaching the NIC to the BIG-IP B instance:

    Field Value
    Description HA-B
    Subnet 10.0.3.0/24
    Private IP 10.0.3.185
    Security groups InternalTraffic

  9. Reboot both BIG-IP VEs so that they can register the new NICs. To do this, right-click each instance in the Instances list and choose Instance State -> Reboot.

Create VLANs for HA communication

You must create a VLAN on each BIG-IP VE. The two BIG-IP VEs will use this VLAN for high availability communication with each other.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Network -> VLANs. The VLAN List screen opens.

  3. Click Create and fill in the appropriate fields for the HA VLAN.

    Field Value
    Name HA
    Interface 1.3
    Tagging Untagged

  4. Click Finished.

  5. Now log in to the BIG-IP Configuration utility on BIG-IP B.

  6. Repeat this task, using the same name for the VLAN:

    Field Value
    Name HA
    Interface 1.3
    Tagging Untagged

  7. Click Finished.

After you complete this task, each BIG-IP VE has a VLAN for high availability communications that corresponds to the HA subnet in your Amazon Virtual Private Cloud (VPC).

Create static self IP addresses for the HA VLANs

Each BIG-IP VE needs a static self IP address to send failover communications to the other BIG-IP VE. This self IP address must match the primary private IP address of the instance’s network interface for the HA subnet.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Network -> Self IPs.

  3. Click Create and populate the appropriate fields.

    Field Value
    Name HASelfIP_A
    IP Address 10.0.3.96
    Netmask 255.255.255.0
    VLAN/Tunnel HA
    Port Lockdown Allow All
    Traffic Group traffic-group-local-only

  4. Click Finished.

  5. Now log in to the BIG-IP Configuration utility on BIG-IP B.

  6. Repeat this task, specifying these values:

    Field Value
    Name HAselfIPB
    IP Address 10.0.3.185
    Netmask 255.255.255.0
    VLAN/Tunnel HA
    Port Lockdown Allow All
    Traffic Group traffic-group-local-only

  1. Click Finished.

The two BIG-IP VEs can now monitor each other’s availability status through the HA VLAN.

Establish trust between the BIG-IP VEs

Before joining a Sync-Failover device group, both BIG-IP VEs must authenticate each others’ certificates to create trust.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Device Management -> Device Trust, and then select Peer List.

  3. Click Add.

  4. For the IP address, type the management address for BIG-IP B, 10.0.0.201.

    This is the primary private IP address associated with BIG-IP B’s management subnet.

  5. Type the administrative user name (admin).

  6. Click Retrieve Device Information.

    BIG-IP A discovers BIG-IP B and displays information about it.

  7. Confirm that BIG-IP B’s certificate is correct.

  8. Confirm that the management IP address and name of BIG-IP B are correct.

  9. Click Finished.

BIG-IP A and BIG-IP B now trust each other.

Specify config sync, failover, and mirroring addresses

Each BIG-IP VE needs to synchronize its configuration with and assess the health of the other BIG-IP VE.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.
  2. On the Main tab, click Device Management -> Devices.
  3. In the Name column, click BIG-IP A.
  4. From the Device Connectivity menu, choose ConfigSync.
  5. For the Local Address setting, select the static self IP address for BIG-IP A’s internal VLAN, 10.0.2.200, and click Update.
  6. From the Device Connectivity menu, choose Failover Network.
  7. For the Failover Unicast Configuration settings, click Add and specify the static self IP address for BIG-IP A’s HA VLAN, 10.0.3.96.
  8. Click Finished.

Now log in to BIG-IP B.

  1. On the Main tab, click Device Management -> Devices.
  2. In the Name column, click BIG-IP B.
  3. From the Device Connectivity menu, choose ConfigSync.
  4. For the Local Address setting, select the static self IP address for BIG-IP B’s internal VLAN, 10.0.2.201, and click Update.
  5. From the Device Connectivity menu, choose Failover Network.
  6. For the Failover Unicast Configuration settings, click Add and specify the static self IP address for BIG-IP B’s HA VLAN, 10.0.3.185.
  7. Click Finished.

Now each BIG-IP VE can use the IP addresses of the other BIG-IP VE to sync its configuration and fail over.

Create a Sync-Failover device group

You must put the two BIG-IP-IP VEs into a Sync-Failover device group. If an active BIG-IP VE in the Sync-Failover device group becomes unavailable, its configuration objects fail over to the other BIG-IP VE and traffic processing resumes.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.
  2. On the Main tab, click Device Management -> Device Groups.
  3. On the Device Groups list screen, click Create.
  4. Type a name for the device group, like bigip_ve_dg.
  5. Select the device group type Sync-Failover.
  6. In the Configuration area of the screen, select both BIG-IP VEs from the Available list and click the Move button.
  7. The BIG-IP VEs are now in the Includes list.
  8. Select the Network Failover check box.
  9. Click Finished.

You now have a Sync-Failover device group that contains both BIG-IP VEs.

Sync the BIG-IP configuration to the device group

You must synchronize the BIG-IP configuration data from BIG-IP A to BIG-IP B. This data includes the floating virtual IP address, 10.0.1.202.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Device Management -> Overview.

  3. In the Device Groups area of the screen, from the Name column, select the device group you created earlier, such as bigip_ve_dg.

    The screen expands to show a summary and details of the sync status of the device group, as well as a list of the two BIG-IP VEs within the device group.

  4. In the Devices area of the screen, from the Sync Status column, select the device that shows a sync status of Changes Pending.

  5. In the Sync Options area of the screen, select Sync Device to Group.

This syncs the most recent changes on BIG-IP A to the other member of bigip_ve_dg, BIG-IP B.

Trigger failover to the standby BIG-IP VE

Before doing this task, confirm in AWS that both BIG-IP VE instances are running.

You can test your HA configuration by forcing the active BIG-IP VE to fail over to the standby peer and then viewing the HA status of each BIG-IP VE.

  1. Log in to the Configuration utility for both BIG-IP VEs.

    In the upper left corner, BIG-IP A should show a status of ACTIVE, while BIG-IP B shows a status of STANDBY:

    ../_images/bigip_a_active.png
  2. In the AWS Management Console, from the Services menu at the top of the screen, select EC2.

  3. In the Navigation pane, under NETWORK & SECURITY, select Network Interfaces.

    This displays the list of EC2 network interfaces.

  4. Find the secondary private IP address, which you will use for the virtual IP address (10.0.1.202); this is the private IP address associated with BIG-IP A’s external interface:

    ../_images/trigger_IP_1.png
  5. On the active BIG-IP VE (BIG-IP A), from the Main tab, click Device Management -> Traffic Groups.

  6. To the left of traffic-group-1, select the check box.

  7. Click Force to Standby.

    A confirmation message appears.

  8. Click Force to Standby again.

    In the upper left corner of the BIG-IP Configuration utility, BIG-IP A now shows a status of STANDBY, while BIG-IP B shows a status of ACTIVE:

    ../_images/bigip_b_active.png
  9. Now view the AWS list of network interfaces and find the secondary private IP address again. You can see that the IP address floated to BIG-IP B’s external interface during failover:

    ../_images/trigger_IP_2.png

Troubleshooting the HA configuration

There are a few things you can do if failover is not working:

  • Confirm that the Port Lockdown setting on each self IP address is Allow All.
  • Confirm that you assigned an IAM role to both instances, and that it has the appropriate security policy assigned to it. If you did not assign a role, you can assign it later, or create an IAM user instead and enter the user’s keys into BIG-IP VE. For more information, see the topic Use an IAM user instead of an IAM role.
  • For the external, internal, and HA VLANs, confirm that the interface assigned to each VLAN matches the device index assigned to the corresponding subnet. For example, the internal subnet in AWS should have a device index of eth2, and the internal VLAN in the BIG-IP software should have interface 1.2 assigned to it.
  • Check the log messages by using SSH to log in to the BIG-IP VEs. At the system prompt, type the command tail -n 20 /var/log/ltm. This shows the most recent twenty rows of log messages.
  • Confirm that the two instances show the same date and time.

If none of the above solves the problem, use the BIG-IP Configuration utility to do the following:

  1. Delete the peer authority in the local trust domain.
  2. Remove the BIG-IP VEs from the device group and then delete the empty device group.
  3. On BIG-IP A, re-establish trust with BIG-IP B, specifying BIG-IP B’s management address, 10.0.0.201.
  4. Re-create the Sync-Failover device group with the Network Failover setting enabled.
  5. On BIG-IP A, sync the configuration to the device group (in this case, BIG-IP B).

High availability networking objects

If you are having issues with your HA configuration, ensure you have all of these object properly configured.

In AWS, a VPC with:

  • Network address translation (NAT)
  • A subnet for the management, external, internal, and HA networks
  • A security group for each subnet
  • A route table entry to provide Internet access for the management and external subnets

A running instance of BIG-IP VE (called BIG-IP A) with the following:

Location Object Details
AWS NICs
  • mgmt_A, eth0, 10.0.0.200
  • external_A, eth1, 10.0.1.200
  • internal_A, eth2, 10.0.2.200
  • HA, eth3, 10.0.3.96
AWS Elastic IP For the management interface, an Elastic IP (EIP) address, for example 52.x.x.x
AWS Secondary Private IP address For the virtual server, a secondary private IP address attached to NIC external_A: 10.0.1.202
BIG-IP VE VLANs
  • external VLAN interface: 1.1
  • internal VLAN interface: 1.2
  • HA VLAN interface: 1.3
BIG-IP VE Self IP addresses
  • External: 10.0.1.200
  • Internal: 10.0.2.200
  • HA: 10.0.3.96
BIG-IP VE Virtual server 10.0.1.202
BIG-IP VE Load balancing pool HA_pool

A running instance of BIG-IP VE (called BIG-IP B) with the following:

Location Object Details
AWS NICs
  • mgmt_B, eth0, 10.0.0.201
  • external_B, eth1, 10.0.1.201
  • internal_B, eth2, 10.0.2.201
  • HA, eth3, 10.0.3.185
AWS Elastic IP For the management interface, an Elastic IP (EIP) address, for example 52.x.x.x
BIG-IP VE VLANs
  • external VLAN interface: 1.1
  • internal VLAN interface: 1.2
  • HA VLAN interface: 1.3
BIG-IP VE Self IP addresses
  • External: 10.0.1.201
  • Internal: 10.0.2.201
  • HA: 10.0.3.185

Create an AWS IAM policy and role for HA

For HA to work, AWS must communicate with BIG-IP VE. To enable this communication, create a role and assign it to BIG-IP VE instances.

  1. First, create a policy to assign to the role. In the AWS Management Console, from the Services menu at the top of the screen, select IAM.

  2. In the Navigation pane, under Details, select Policies.

  3. Click Create Policy.

  4. By Create Your Own Policy, click Select.

  5. Enter this text in the Policy Document field.

    {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
              "ec2:describeinstancestatus",
              "ec2:describenetworkinterfaces",
              "ec2:assignprivateipaddresses"
              ],
              "Resource": "*"
          }
              ]
    }
    
  6. Click Create Policy.

  7. Now, create a role and assign the policy to it. In the Navigation pane, under Details, select Roles.

  8. Click Create New Role.

  9. Type a name and click Next Step.

  10. Under AWS Service Roles, next to Amazon EC2, click Select.

  11. Select the policy you created and click Next Step.

  12. Click Create Role.

  13. Assign the role to the instance. From the Services menu, click EC2.

  14. Click Running Instances.

  15. Right-click the BIG-IP VE instance and choose Instance Settings > Attach/Replace IAM Role.

  16. Repeat step 15 for the other BIG-IP VE instance.

Use an IAM user instead of an IAM role

For BIG-IP VE and AWS to communicate, an IAM user or role with sufficient permission must exist in AWS. If you used an IAM role, you assigned it when you deployed BIG-IP VE. If you prefer, you can use an IAM user instead.

  1. Create an AWS IAM policy.

    1. In the AWS Management Console, from the Services menu at the top of the screen, select IAM.

    2. In the Navigation pane, under Details, select Policies.

    3. Click Create Policy.

    4. By Create Your Own Policy, click Select.

    5. Enter this text in the Policy Document field.

      {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                "ec2:describeinstancestatus",
                "ec2:describenetworkinterfaces",
                "ec2:assignprivateipaddresses"
                ],
                "Resource": "*"
            }
        ]
       }
      
    6. Click Create Policy.

  2. Now assign the policies to an IAM user.

    1. In the Navigation pane, under Details, select Users.

    2. Click Create New Users.

    3. Type a user name, select Generate an access key for each user and then click Create.

    4. Click Download Credentials.

      An access key ID and a secret access key are in a file named credentials.csv.

      Important: AWS downloads these credentials only once, so keep track of them.

    5. Click Close.

    6. In the list of users, click the row for the user.

    7. On the Permissions tab, click Attach Policy.

    8. Select the check box for the policy you created previously.

    9. Click Attach Policy.

  3. Finally, enter the user’s keys into BIG-IP VE.

    1. Log in to the BIG-IP Configuration utility.

    2. On the Main tab, click System -> Configuration -> AWS -> Global Settings.

    3. In the Access Key field, type the access key.

    4. In the Secret Key field, type the secret key.

      ../_images/secret_key.png
    5. Click Update.

The IAM user can now communicate between BIG-IP VE and AWS.

Check the metadata service for IAM role

Use BIG-IP Bash Shell to help check metadata service on both BIG-Ip instances so the IAM role can be applied to the instance. For example:

[admin@ip-10-0-11-72:Active:In Sync] ~ # curl http://169.254.169.254/latest/meta-data/iam/info
{

"Code" : "Success",

"LastUpdated" : "2019-10-02T17:54:43Z",

"InstanceProfileArn" : "arn:aws:iam::XXXXXXXX:instance-profile/my-ha-role",

"InstanceProfileId" : "AIPA5XXXXXXXXXXXXXXX"

}

Check the AWS API is reachable

Use BIG-IP Bash Shell to check AWS API is reachable on both BIG-IP instances, so a failover can occur. For example:

[admin@ip-10-0-11-72:Active:In Sync] ~ # curl -sI https://ec2.us-east-1.amazonaws.com | grep Server
Server: AmazonEC2

Replace the ec2.us-east-1 with your region/zone.