Cloud Docs Home > F5 Public Cloud Integrations Index

Amazon Web Services: High Availability BIG-IP VE

If a BIG-IP VE becomes unavailable for any reason, it can fail over to another BIG-IP VE. In the AWS environment, if the active BIG-IP VE suddenly goes offline, BIG-IP VE drops the active connections. The other BIG-IP VE processes new connections when it becomes active.

The following illustration shows an example of two BIG-IP VE instances in an Amazon VPC. The two BIG-IP VEs are members of a BIG-IP device group, which means that the BIG-IPs trust each other, they synchronize their configurations, and they can fail over to one another.

Each BIG-IP VE has the default floating traffic group, traffic-group-1, that contains a floating virtual IP address. Application traffic is going to the virtual IP address on BIG-IP A. If BIG-IP A goes offline, the virtual IP address becomes active on BIG-IP B and the traffic redirects to it.

../_images/ha1.png

Complete the tasks in this guide to create this deployment.

Alternately, you can use CloudFormation templates to create this deployment. For more information about CloudFormation templates provided by F5, go to https://github.com/F5Networks.

High availability configuration overview

This illustration shows the additional network objects you must create for a typical BIG-IP VE high availability (HA) configuration in AWS.

In this configuration, the BIG-IP VEs continually communicate their availability status to one another through the HA VLAN and the associated static self IP address on each BIG-IP VE.

../_images/ha2.png

To create this configuration, in AWS, you create an HA subnet with primary private IP addresses and network interfaces. Then in BIG-IP VE, you create corresponding objects, represented by the shaded boxes in the diagram.

Task List: Create a second BIG-IP VE instance

Before you can configure high availability (HA), follow the steps in the Amazon Web Services: Multi-NIC Configuration guide to create one BIG-IP VE instance (BIG-IP A) in an Amazon VPC with multiple subnets.

Then complete the following tasks, which are a subset of the tasks in the Multi-NIC Configuration guide, to create a second BIG-IP VE instance (BIG-IP B).

Note: Both BIG-IP VE instances must be in the same availability zone. For HA across availability zones, see the CFTs on https://github.com/F5Networks.

In AWS:

Step Task Description
1 Deploy a BIG-IP VE instance

The second BIG-IP VE should be in the same VPC as the first. Add the extra, external NIC.

  • Management: 10.0.0.201
  • External: 10.0.1.201
2 Enable communication between BIG-IP VE and AWS

An IAM user or role with sufficient policy permissions must exist in AWS. Assign the role to both BIG-IP VE instances, or enter the user’s keys.

To use an IAM role for communication, see this topic. IAM roles work in BIG-IP VE 13.0.0 and later.

To use a IAM user for communication, see this topic.

3 Create an internal network interface (NIC) and attach it to the instance

You created NICs for the management and external subnets when you deployed the instance. You must create an internal NIC and reboot, so BIG-IP VE can recognize the new NIC, and hourly instances can license with F5.

  • Internal: 10.0.2.201
4 Create an Elastic IP address (EIP) for the BIG-IP management interface An EIP address is a publicly-routable address that provides access to the BIG-IP Configuration utility. If the BIG-IP VE reboots, stops, or terminates, the EIP address persists on that NIC.

On the new BIG-IP VE instance (BIG-IP B):

Step Task Description
1 Connect to the BIG-IP VE instance and set the admin password

Before you can license and provision BIG-IP VE, use SSH and your key pair to connect to the instance and set a strong password.

  • In tmsh, type modify auth password admin.
2 Log in and provision BIG-IP VE Log in to the BIG-IP Configuration utility (https://<ElasticIP>) and provision BIG-IP VE. If you chose a BYOL license, you must license BIG-IP VE before provisioning.
3 Create external and internal VLANs

These VLANs correspond to the external and internal subnets in your VPC.

  • External VLAN interface: 1.1
  • Internal VLAN interface: 1.2
4 Create static self IP addresses for the external and internal VLANs

These addresses should match the private IP addresses you assigned to the external and internal subnets in AWS.

  • External: 10.0.1.201
  • Internal: 10.0.2.201

Task List: Configure BIG-IP VE high availability

To set up high availability (HA), create these resources. This is a specific example, which you can use to test an HA configuration.

Step Task Description
1 Create a subnet for HA communication

In AWS, the VPC needs a separate subnet for HA communication between BIG-IP VE instances.

  • HA Subnet: 10.0.3.0/24
2 Create network interfaces (NICs) for the HA subnet

In AWS, create two NICs for HA and attach one to each BIG-IP VE instance.

  • Interface: eth3
3 Create VLANs for HA communication

On each BIG-IP VE, create a VLAN that corresponds to the HA subnet.

  • VLAN: HA
4 Create static self IP addresses for the HA VLANs

On each BIG-IP VE, create a static self IP address used for failover communication. These IP addresses must match the private IP addresses assigned to the HA subnet in AWS.

  • Self IP on BIG-IP A: 10.0.3.96
  • Self IP on BIG-IP B: 10.0.3.185
5 Establish device trust

The BIG-IP VEs must establish trust by exchanging certificates. Use management IP addresses to do this.

  • Management IP on BIG-IP A: 10.0.0.200
  • Management IP on BIG-IP B: 10.0.0.201
6 Specify config sync and failover addresses

These are the static self IP addresses that you want the BIG-IP VEs to use for config sync and failover operations to one another.

Config sync static self IP for internal VLAN:

  • BIG-IP A: 10.0.2.200
  • BIG-IP B: 10.0.2.201

Static self IP for the HA VLAN:

  • BIG-IP A: 10.0.3.96
  • BIG-IP B: 10.0.3.185
7 Create a Sync-Failover device group

BIG-IP VEs in a Sync-Failover device group can sync their configurations and fail over to one another.

  • bigip_ve_dg
8 Synchronize the BIG-IP configuration Log into BIG-IP A and sync its configuration to BIG-IP B.

Create a subnet for HA communication

Each BIG-IP VE instance uses three VPC subnets, for management, external, and internal traffic. Note the availability zone for these subnets (for example, us-west-2a).

Now, in the same availability zone, create a subnet for high availability (HA) communication between the two instances. This subnet corresponds to the BIG-IP VLAN named HA that you will create later on each BIG-IP VE.

  1. In the AWS Management Console, from the Services menu at the top of the screen, select VPC.
  2. In the Navigation pane, under Virtual Private Cloud, select Subnets.
  3. Click Create Subnet.
  4. In the Name tag field, type HA.
  5. In the VPC field, select the VPC.
  6. In the Availability Zone field, select the zone where the other subnets reside.
  7. In the CIDR block field, type 10.0.3.0/24.
  8. Click Yes, Create.

Your VPC should now have four subnets:

  • management: 10.0.0.0
  • external: 10.0.1.0
  • internal: 10.0.2.0
  • HA: 10.0.3.0

Create HA network interfaces

Each of your BIG-IP VE instances should have three network interfaces, one per subnet (management, external, and internal). Now create another network interface for each instance and associate it with the HA subnet.

  1. In the AWS Management Console, from the Services menu at the top of the screen, select EC2.

  2. In the Navigation pane, under NETWORK & SECURITY, select Network Interfaces.

  3. Click Create Network Interface and populate the appropriate fields.

    Field Value
    Description HA-A
    Subnet 10.0.3.0/24
    Private IP 10.0.3.96
    Security groups InternalTraffic

    Note: You do not need to create a separate security group for the HA network interfaces.

  4. Click Yes, Create.

    AWS adds the network interface to the list.

  5. Update the name in the list to HA-A.

  6. Right-click the new network interface and select Attach.

  7. From the Instance ID list, select the instance for BIG-IP A and click Attach.

  8. Repeat this task for BIG-IP B, using these values and attaching the NIC to the BIG-IP B instance:

    Field Value
    Description HA-B
    Subnet 10.0.3.0/24
    Private IP 10.0.3.185
    Security groups InternalTraffic

  9. Reboot both BIG-IP VEs so that they can register the new NICs. To do this, right-click each instance in the Instances list and choose Instance State -> Reboot.

Create VLANs for HA communication

You must create a VLAN on each BIG-IP VE. The two BIG-IP VEs will use this VLAN for high availability communication with each other.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Network -> VLANs. The VLAN List screen opens.

  3. Click Create and fill in the appropriate fields for the HA VLAN.

    Field Value
    Name HA
    Interface 1.3
    Tagging Untagged

  4. Click Finished.

  5. Now log in to the BIG-IP Configuration utility on BIG-IP B.

  6. Repeat this task, using the same name for the VLAN:

    Field Value
    Name HA
    Interface 1.3
    Tagging Untagged

  7. Click Finished.

After you complete this task, each BIG-IP VE has a VLAN for high availability communications that corresponds to the HA subnet in your Amazon Virtual Private Cloud (VPC).

Create static self IP addresses for the HA VLANs

Each BIG-IP VE needs a static self IP address to send failover communications to the other BIG-IP VE. This self IP address must match the primary private IP address of the instance’s network interface for the HA subnet.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Network -> Self IPs.

  3. Click Create and populate the appropriate fields.

    Field Value
    Name HASelfIP_A
    IP Address 10.0.3.96
    Netmask 255.255.255.0
    VLAN/Tunnel HA
    Port Lockdown Allow All
    Traffic Group traffic-group-local-only

  4. Click Finished.

  5. Now log in to the BIG-IP Configuration utility on BIG-IP B.

  6. Repeat this task, specifying these values:

    Field Value
    Name HAselfIPB
    IP Address 10.0.3.185
    Netmask 255.255.255.0
    VLAN/Tunnel HA
    Port Lockdown Allow All
    Traffic Group traffic-group-local-only

  1. Click Finished.

The two BIG-IP VEs can now monitor each other’s availability status through the HA VLAN.

Establish trust between the BIG-IP VEs

Before joining a Sync-Failover device group, both BIG-IP VEs must authenticate each others’ certificates to create trust.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Device Management -> Device Trust, and then select Peer List.

  3. Click Add.

  4. For the IP address, type the management address for BIG-IP B, 10.0.0.201.

    This is the primary private IP address associated with BIG-IP B’s management subnet.

  5. Type the administrative user name (admin).

  6. Click Retrieve Device Information.

    BIG-IP A discovers BIG-IP B and displays information about it.

  7. Confirm that BIG-IP B’s certificate is correct.

  8. Confirm that the management IP address and name of BIG-IP B are correct.

  9. Click Finished.

BIG-IP A and BIG-IP B now trust each other.

Specify config sync, failover, and mirroring addresses

Each BIG-IP VE needs to synchronize its configuration with and assess the health of the other BIG-IP VE.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.
  2. On the Main tab, click Device Management -> Devices.
  3. In the Name column, click BIG-IP A.
  4. From the Device Connectivity menu, choose ConfigSync.
  5. For the Local Address setting, select the static self IP address for BIG-IP A’s internal VLAN, 10.0.2.200, and click Update.
  6. From the Device Connectivity menu, choose Failover Network.
  7. For the Failover Unicast Configuration settings, click Add and specify the static self IP address for BIG-IP A’s HA VLAN, 10.0.3.96.
  8. Click Finished.

Now log in to BIG-IP B.

  1. On the Main tab, click Device Management -> Devices.
  2. In the Name column, click BIG-IP B.
  3. From the Device Connectivity menu, choose ConfigSync.
  4. For the Local Address setting, select the static self IP address for BIG-IP B’s internal VLAN, 10.0.2.201, and click Update.
  5. From the Device Connectivity menu, choose Failover Network.
  6. For the Failover Unicast Configuration settings, click Add and specify the static self IP address for BIG-IP B’s HA VLAN, 10.0.3.185.
  7. Click Finished.

Now each BIG-IP VE can use the IP addresses of the other BIG-IP VE to sync its configuration and fail over.

Create a Sync-Failover device group

You must put the two BIG-IP-IP VEs into a Sync-Failover device group. If an active BIG-IP VE in the Sync-Failover device group becomes unavailable, its configuration objects fail over to the other BIG-IP VE and traffic processing resumes.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.
  2. On the Main tab, click Device Management -> Device Groups.
  3. On the Device Groups list screen, click Create.
  4. Type a name for the device group, like bigip_ve_dg.
  5. Select the device group type Sync-Failover.
  6. In the Configuration area of the screen, select both BIG-IP VEs from the Available list and click the Move button.
  7. The BIG-IP VEs are now in the Includes list.
  8. Select the Network Failover check box.
  9. Click Finished.

You now have a Sync-Failover device group that contains both BIG-IP VEs.

Sync the BIG-IP configuration to the device group

You must synchronize the BIG-IP configuration data from BIG-IP A to BIG-IP B. This data includes the floating virtual IP address, 10.0.1.202.

Note: Do this task on BIG-IP A only.

  1. Log in to the BIG-IP Configuration utility on BIG-IP A.

  2. On the Main tab, click Device Management -> Overview.

  3. In the Device Groups area of the screen, from the Name column, select the device group you created earlier, such as bigip_ve_dg.

    The screen expands to show a summary and details of the sync status of the device group, as well as a list of the two BIG-IP VEs within the device group.

  4. In the Devices area of the screen, from the Sync Status column, select the device that shows a sync status of Changes Pending.

  5. In the Sync Options area of the screen, select Sync Device to Group.

This syncs the most recent changes on BIG-IP A to the other member of bigip_ve_dg, BIG-IP B.

Trigger failover to the standby BIG-IP VE

Before doing this task, confirm in AWS that both BIG-IP VE instances are running.

You can test your HA configuration by forcing the active BIG-IP VE to fail over to the standby peer and then viewing the HA status of each BIG-IP VE.

  1. Log in to the Configuration utility for both BIG-IP VEs.

    In the upper left corner, BIG-IP A should show a status of ACTIVE, while BIG-IP B shows a status of STANDBY:

    ../_images/bigip_a_active.png

  2. In the AWS Management Console, from the Services menu at the top of the screen, select EC2.

  3. In the Navigation pane, under NETWORK & SECURITY, select Network Interfaces.

    This displays the list of EC2 network interfaces.

  4. Find the secondary private IP address, which you will use for the virtual IP address (10.0.1.202); this is the private IP address associated with BIG-IP A’s external interface:

    ../_images/trigger_IP_1.png

  5. On the active BIG-IP VE (BIG-IP A), from the Main tab, click Device Management -> Traffic Groups.

  6. To the left of traffic-group-1, select the check box.

  7. Click Force to Standby.

    A confirmation message appears.

  8. Click Force to Standby again.

    In the upper left corner of the BIG-IP Configuration utility, BIG-IP A now shows a status of STANDBY, while BIG-IP B shows a status of ACTIVE:

    ../_images/bigip_b_active.png

  9. Now view the AWS list of network interfaces and find the secondary private IP address again. You can see that the IP address floated to BIG-IP B’s external interface during failover:

    ../_images/trigger_IP_2.png

Troubleshooting the HA configuration

There are a few things you can do if failover is not working:

  • Confirm that the Port Lockdown setting on each self IP address is Allow All.
  • Confirm that you assigned an IAM role to both instances, and that it has the appropriate security policy assigned to it. If you did not assign a role, you can assign it later, or create an IAM user instead and enter the user’s keys into BIG-IP VE. For more information, see the topic Use an IAM user instead of an IAM role.
  • For the external, internal, and HA VLANs, confirm that the interface assigned to each VLAN matches the device index assigned to the corresponding subnet. For example, the internal subnet in AWS should have a device index of eth2, and the internal VLAN in the BIG-IP software should have interface 1.2 assigned to it.
  • Check the log messages by using SSH to log in to the BIG-IP VEs. At the system prompt, type the command tail -n 20 /var/log/ltm. This shows the most recent twenty rows of log messages.
  • Confirm that the two instances show the same date and time.

If none of the above solves the problem, use the BIG-IP Configuration utility to do the following:

  1. Delete the peer authority in the local trust domain.
  2. Remove the BIG-IP VEs from the device group and then delete the empty device group.
  3. On BIG-IP A, re-establish trust with BIG-IP B, specifying BIG-IP B’s management address, 10.0.0.201.
  4. Re-create the Sync-Failover device group with the Network Failover setting enabled.
  5. On BIG-IP A, sync the configuration to the device group (in this case, BIG-IP B).

High availability networking objects

If you are having issues with your HA configuration, ensure you have all of these object properly configured.

In AWS, a VPC with:

  • Network address translation (NAT)
  • A subnet for the management, external, internal, and HA networks
  • A security group for each subnet
  • A route table entry to provide Internet access for the management and external subnets

A running instance of BIG-IP VE (called BIG-IP A) with the following:

Location Object Details
AWS NICs
  • mgmt_A, eth0, 10.0.0.200
  • external_A, eth1, 10.0.1.200
  • internal_A, eth2, 10.0.2.200
  • HA, eth3, 10.0.3.96
AWS Elastic IP For the management interface, an Elastic IP (EIP) address, for example 52.x.x.x
AWS Secondary Private IP address For the virtual server, a secondary private IP address attached to NIC external_A: 10.0.1.202
BIG-IP VE VLANs
  • external VLAN interface: 1.1
  • internal VLAN interface: 1.2
  • HA VLAN interface: 1.3
BIG-IP VE Self IP addresses
  • External: 10.0.1.200
  • Internal: 10.0.2.200
  • HA: 10.0.3.96
BIG-IP VE Virtual server 10.0.1.202
BIG-IP VE Load balancing pool HA_pool

A running instance of BIG-IP VE (called BIG-IP B) with the following:

Location Object Details
AWS NICs
  • mgmt_B, eth0, 10.0.0.201
  • external_B, eth1, 10.0.1.201
  • internal_B, eth2, 10.0.2.201
  • HA, eth3, 10.0.3.185
AWS Elastic IP For the management interface, an Elastic IP (EIP) address, for example 52.x.x.x
BIG-IP VE VLANs
  • external VLAN interface: 1.1
  • internal VLAN interface: 1.2
  • HA VLAN interface: 1.3
BIG-IP VE Self IP addresses
  • External: 10.0.1.201
  • Internal: 10.0.2.201
  • HA: 10.0.3.185

Create an AWS IAM policy and role for HA

For HA to work, AWS must communicate with BIG-IP VE. To enable this communication, create a role and assign it to BIG-IP VE instances.

  1. First, create a policy to assign to the role. In the AWS Management Console, from the Services menu at the top of the screen, select IAM.

  2. In the Navigation pane, under Details, select Policies.

  3. Click Create Policy.

  4. By Create Your Own Policy, click Select.

  5. Enter this text in the Policy Document field.

    {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
              "ec2:describeinstancestatus",
              "ec2:describenetworkinterfaces",
              "ec2:assignprivateipaddresses"
              ],
              "Resource": "*"
          }
              ]
    }
    
  6. Click Create Policy.

  7. Now, create a role and assign the policy to it. In the Navigation pane, under Details, select Roles.

  8. Click Create New Role.

  9. Type a name and click Next Step.

  10. Under AWS Service Roles, next to Amazon EC2, click Select.

  11. Select the policy you created and click Next Step.

  12. Click Create Role.

  13. Assign the role to the instance. From the Services menu, click EC2.

  14. Click Running Instances.

  15. Right-click the BIG-IP VE instance and choose Instance Settings > Attach/Replace IAM Role.

  16. Repeat step 15 for the other BIG-IP VE instance.

Use an IAM user instead of an IAM role

For BIG-IP VE and AWS to communicate, an IAM user or role with sufficient permission must exist in AWS. If you used an IAM role, you assigned it when you deployed BIG-IP VE. If you prefer, you can use an IAM user instead.

  1. Create an AWS IAM policy.

    1. In the AWS Management Console, from the Services menu at the top of the screen, select IAM.

    2. In the Navigation pane, under Details, select Policies.

    3. Click Create Policy.

    4. By Create Your Own Policy, click Select.

    5. Enter this text in the Policy Document field.

      {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                "ec2:describeinstancestatus",
                "ec2:describenetworkinterfaces",
                "ec2:assignprivateipaddresses"
                ],
                "Resource": "*"
            }
        ]
       }
      
    6. Click Create Policy.

  2. Now assign the policies to an IAM user.

    1. In the Navigation pane, under Details, select Users.

    2. Click Create New Users.

    3. Type a user name, select Generate an access key for each user and then click Create.

    4. Click Download Credentials.

      An access key ID and a secret access key are in a file named credentials.csv.

      Important: AWS downloads these credentials only once, so keep track of them.

    5. Click Close.

    6. In the list of users, click the row for the user.

    7. On the Permissions tab, click Attach Policy.

    8. Select the check box for the policy you created previously.

    9. Click Attach Policy.

  3. Finally, enter the user’s keys into BIG-IP VE.

    1. Log in to the BIG-IP Configuration utility.

    2. On the Main tab, click System -> Configuration -> AWS -> Global Settings.

    3. In the Access Key field, type the access key.

    4. In the Secret Key field, type the secret key.

      ../_images/secret_key.png
    5. Click Update.

The IAM user can now communicate between BIG-IP VE and AWS.