Lab 1: High Availability & Efficient Load Distribution

Business problem : AI pipelines need consistent, high‑throughput access to S3-compatible storage. Wiring clients directly to specific storage nodes creates tight coupling and operational risk: a single overloaded node throttles the entire pipeline.

Technical problem : Without a delivery layer, clients must pick a node, handle retries/failover, and live with uneven utilization (hot spots) and brittle endpoints.

Solution with BIG‑IP LTM : Expose a single, resilient virtual endpoint. Behind this VIP, BIG‑IP intelligently distributes S3 traffic across all healthy MinIO nodes and lets you scale by simply adding/removing pool members—no client changes required. Use Least Connections to smooth throughput for S3 workloads.

Following the tasks in the prior Introduction Section, you should now be able to access the UDF lab environment.

Task 1: Review the Lab Environment

These values align with the UDF topology. Keep them unchanged unless your environment differs.

Component	Purpose	Where to access
MinIO Cluster‑1 (direct)	Baseline test without BIG‑IP	WARP parameters: 10.1.10.100:9000
BIG‑IP VIP for Cluster‑1	Single front door with LTM load	WARP parameters: 10.1.40.160:9000
MinIO Console	Review node-level metrics	UDF → Cluster1-Node1 → Access → UI
BIG‑IP TMUI	Verify pools/members & methods	UDF → BIG‑IP → Access → TMUI

Task 2: Baseline: Send traffic directly to MinIO (no BIG‑IP)

The following steps will validate access to the application via a web browser, review the Performance Monitoring dashboard, and gather request details.

Quick aside, what is Warp? It is a high-performance, open-source benchmarking tool designed to measure and analyze the throughput and latency of S3-compatible object storage systems.

Warp simulates real-world workloads—such as PUT (uploads), GET (downloads), LIST, and DELETE operations—to evaluate performance, plan capacity, and identify bottlenecks.

The labs offer an easy-to-use graphical front end for Warp, to avoid needing to issue command-line actions.

Open MinIO WARP (UDF → Components → Traffic‑Gen → Access → Firefox). The credentials are under lab Documentation tab (admin/admin). If presented with Firefox "Restoring Pages" message, choose "Restore Session" button. As well, permit the pop-up to allow access to clipboard. Select the cluster‑1 profile. Select all 3 buckets, when selected for use they will appear in bright orange. Set Duration to 3 minutes and Concurrency to 20 threads. Conncurrency refers to parallel S3 transactions. In WARP Parameters, set Endpoint to 10.1.10.100:9000. Click Run Benchmark.

Open MinIO cluster‑1 Console (UDF → Cluster1‑Node1 → Access → UI). Login: minioadmin / minioadmin Observe that there are 4 AIStor servers spread across 2 clusters, however data charts require normally 30 minutes or more to popluate so expect no traffic on the right-hand chart. REMINDER of ABOVE You will not see the charts below filling for at a number of minutes, data is added on 15-minute increments through the hour (0:00, 0:15, 0:45, 1:00). Data will appear after initial write.

Click on the arrow next to time to first byte, in the lower right of the screen. Observe that once the charts populate, only traffic will be received by the first AIStor, at address 10.1.10.100 port 9000. This traffic will task one server, creating a hot spot of load.

Why this matters: Clients that target a single node are brittleutilization is uneven and scalability suffers.

Task 3: Baseline: Steer (proxy) the Same Workload through BIG-IP

In WARP, switch target to the BIG‑IP VIP profile BigIP‑cluster‑1. Keep all three buckets active. Set Duration: 5 minutes (300 seconds), Concurrency: 20 threads. In WARP Parameters, set Endpoint to 10.1.40.160:9000 (as we are now targeting a virtual server). Click Run Benchmark Log into BIG-IP TMU: Local Traffic → Pools → Cluster‑1. Confirm 3 pool members are present initially Click both Members and Statistics tabs. Important: Observe how the load balancing method has been set to "Least Connections". Note due to short run durations, summary analytics may not have appeared in the AIStor dashboard view yet.

Confirm the WARP S3 load generator has run to completions, traffic settings can be seen below. The AISTor GUI will likley not populate for a few more minutes, but later when checked will look like the AIStor screenshot provided below.

Expected outcome: Traffic is distributed across the three nodes behind the VIP.

Load‑balancing method: This pool is configured for Least Connections, recommended for S3‑style workloads to reduce request skew.

Task 4: Scale out easily: add the 4th MinIO AIStor node to the pool

In BIG‑IP TMUI: Local Traffic → Pools → Cluster‑1 → Members. Add a New member and choose Node List menu.

Click Add.... -> Node List -> select cl1-nd4 Set Service Port to 9000 and click Finished Note Health checks for the new member will drive the LED from blue to green (ready) shortly

Re-run the WARP workload targetting the BIG-IP Virtual IP (VIP 10.1.40.160:9000) Note The 4th AIStor node begins processing traffic immediately. All nodes now share load

Key Takeaway There were no client changes and S3 applications still continue to talk to the same VIP; topology changes are absorbed by BIG‑IP at the dataplane.

Task 5: Verify the load‑balancing method & pool health

The following steps will guide you through the rich, visual metrics presented for BIG-IP through the AST dashboards powered by Grafana. We will verify the load balancing method and pool health.

In BIG‑IP TMUI: Local Traffic → Pools → Cluster‑1. As good practice, confirm Load Balancing Method: Least Connections. This is seen on the Pool Members list screen. Check Members tab: All members green (up) with active connections. Use the AST tool (to review the Dashboards) UDF -> AST -> Access -> Grafana. Login as admin / admin, when prompted to change password simply retain the value as admin In AST, choose Dashboards - BigIP - Device - Device Pools look at the key metrics, such as Active Pool Connections. For "Pool" in top menu, adjust to "Cluster-1" and examine last 15 minutes. Click on the "3 dots" menu → View on any graphical widget to see the full panel. Click "Refresh" often.

The charts indicate an even distribution for MinIO AIStor nodes, hot spots have been successfully avoided.

Validation

MinIO AIStor Console (Main Screen → Data -> arrow next to Time to First Byte): when populated with data points shows very close to even, per‑node traffic once proxied via VIP (some variability is expected / OK).

BIG‑IP Pool Stats: show all 4 members up with active connections.

End‑user impact: Client endpoint unchanged; backend scaling is transparent.

Troubleshooting

WARP can't connect

-> Re‑check endpoint (10.1.10.100:9000 direct vs 10.1.40.160:9000 VIP). -> Ensure the http/https scheme matches your setup (WARP must use the correct protocol).

MinIO AIStore UI Metrics don't update

-> the page isn't real‑time, hit "Refresh" with last 4 hours timerange, towards end of lab to see data

A pool member is down (red)

-> Verify the MinIO node process. -> Review the pool's Monitor and node address/port.

Skew remains after adding cl1‑nd4

-> Confirm the new member is enabled and passing its monitor. -> Ensure Least Connections is applied (not Round Robin).

Clean-Up (Optional)

Stop any running WARP tests.

If you temporarily disabled/enabled members, restore their original state.

Leave Cluster‑1 with 4 active members for Labs 2 and 3.

What You Learned - Value of BIG-IP LTM and AIStor

Decoupling Clients connect to one VIP; storage topology can evolve freely.

Efficiency Least Connections smooths throughput and reduces hotspots.

Elasticity Scale by adding/removing pool members without client changes.

Operational simplicity Traffic engineering lives at the dataplane, not in every client.

End of Lab 1: This concludes Lab 1. In this lab you ran high rate S3 loads against a single MinIO AIStor server, leaving other AIStor instances unused. You then adjusted the load generator to use a virtual server with a virtual IP address on the F5 BIG-IP. An origin pool corresponding to three AIStor instances allowed traffic to be spread, per the least connections approach, to all healthy nodes. A fourth AIStor instance was added to the pool, exercising all nodes equally and not requiring any client side adjustments, a major benefit in scenarios with hundreds or possibly thousands of clients.