OcNOS routing software¶

Overview¶

OcNOS is the routing software that runs the control plane for BGP routing in F5 BIG-IP Next for Kubernetes. Developed by IP Infusion, OcNOS manages all routing protocols, establishes BGP sessions, exchanges routes with the fabric, and distributes reachability information to Traffic Management Microkernel (TMM) for packet forwarding.

You don’t interact with OcNOS directly. Instead, you define your routing intent using Kubernetes Custom Resource Definitions (CRDs), and the Cloud-Native Edge (CNE) controller renders and delivers the configuration to OcNOS automatically. OcNOS runs inside the f5-tmm-routing container alongside Traffic Management Microkernel TMM in each f5-tmm pod.

What OcNOS does¶

OcNOS serves as the brain of the routing system. While TMM handles the data plane (forwarding packets, VXLAN encapsulation and decapsulation, and VRF interface management), OcNOS handles the control plane:

Function	Description
BGP session management	Establishes and maintains BGP sessions with fabric peers. Handles OPEN, KEEPALIVE, NOTIFICATION, and UPDATE messages.
Route advertisement	Advertises local routes (pod CIDRs, service CIDRs, virtual addresses) to BGP peers.
Route learning	Receives routes from BGP peers and installs them in the routing table. Sends learned routes to TMM through tmrouted.
EVPN route exchange	Distributes Route Type 5 (IP prefix) information across the fabric.
BGP Unnumbered	Establishes BGP sessions using IPv6 link-local addresses for simplified fabric peering. Relies on TMM and tmrouted for neighbor discovery.
BFD	Runs the `oamd` daemon for Bidirectional Forwarding Detection (BFD) session management, which provides fast failure detection on BGP links.
VRF awareness	Maintains separate routing tables for each Virtual Routing and Forwarding (VRF) instance, which provides tenant isolation at the control plane level.

What OcNOS does not do¶

OcNOS does not handle packet forwarding. TMM owns the data plane and is responsible for:

Forwarding packets based on routes learned from OcNOS
VXLAN tunnel encapsulation and decapsulation
VRF interface creation in the Linux kernel
Router Advertisement (RA) exchange for BGP Unnumbered neighbor discovery

Architecture¶

OcNOS runs inside the f5-tmm-routing container in each f5-tmm pod. It communicates with other components through the following channels:

OcNOS receives its configuration through an automated pipeline. When you create a GlobalRoutingConfig or RoutingConfig resource, the CNE controller combines the values from your CRD with the referenced RoutingTemplate to produce OcNOS CLI commands. The controller then delivers these rendered commands to the f5-tmm-routing container through gRPC secured with mutual TLS (mTLS). Inside the container, the grpcconf_watcher process applies the commands to OcNOS through the cmlsh CLI interface.

Channel	From	To	What it carries
gRPC (mTLS)	CNE controller	`grpcconf_watcher`	Rendered CLI configuration payloads
CLI (`cmlsh`)	`grpcconf_watcher`	OcNOS	Applies the configuration to the routing daemon
Netlink	OcNOS	tmrouted	Learned routes (FDB entries, ARP entries, dynamic routes)
Netlink	tmrouted	OcNOS	VRF configuration, VLAN bindings, interface addresses, RA data
Protobuf	tmrouted	TMM	Learned routes for forwarding table updates
Protobuf	TMM	tmrouted	VRF state, VLAN state, RA neighbor data

Supported protocols¶

Protocol	Description
BGP	Border Gateway Protocol for exchanging routing information with fabric peers. Supports eBGP and iBGP, numbered and unnumbered modes.
EVPN	Ethernet VPN as a control plane for VXLAN. Distributes MAC/IP and IP prefix reachability across VTEPs using BGP address family `l2vpn-evpn`.
BFD	Bidirectional Forwarding Detection for fast link failure detection. The `oamd` daemon manages BFD sessions; the TMM BFD Proxy handles BFD packets on the data plane.

How OcNOS is configured¶

All configuration flows through Kubernetes CRDs and the RoutingTemplate.

Configuration workflow¶

What the RoutingTemplate produces¶

The RoutingTemplate is a Go template that produces OcNOS CLI commands. When the controller renders the template, it substitutes values from your CRDs (such as the ASN, neighbor IPs, address families, and VNI) into the template body. The output is a complete CLI configuration that OcNOS can apply.

Example: A simple RoutingTemplate and its rendered output

Template:

apiVersion: k8s.f5net.com/v1alpha1
kind: RoutingTemplate
metadata:
name: example-template
spec:
body: |
router bgp {{ .global.spec.bgp.startASN }}
bgp router-id {{ $routerId }}
{{- range $nbr := .global.spec.bgp.neighbors }}
neighbor {{ or $nbr.ip $nbr.interface }} remote-as {{ $nbr.remoteAs }}
{{- end }}
{{- range $nbr := .global.spec.bgp.neighbors }}
{{- if $nbr.families }}
{{- range $fam := $nbr.families }}
address-family {{ $fam }}
neighbor {{ or $nbr.ip $nbr.interface }} activate
exit-address-family
{{- end }}
{{- else }}
address-family ipv4-unicast
neighbor {{ or $nbr.ip $nbr.interface }} activate
exit-address-family
{{- end }}
{{- end }}
exit

Rendered output (for a TMM pod with ASN 65001 and router ID 10.10.10.1):

router bgp 65001
bgp router-id 10.10.10.1
neighbor 198.51.100.10 remote-as 65110
address-family ipv4 unicast
neighbor 198.51.100.10 activate
exit-address-family
exit

Common OcNOS configuration blocks¶

The following OcNOS configuration blocks are commonly included in RoutingTemplate definitions. You don’t write these directly — you include them in the Go template body.

VXLAN and EVPN enablement¶

nvo vxlan enable
nvo vxlan irb

These commands enable VXLAN Network Virtualization Overlay (NVO) and Integrated Routing and Bridging (IRB) mode, which allows both L2 and L3 forwarding on VXLAN interfaces.

BGP neighbor configuration¶

router bgp 65001
neighbor 198.51.100.10 remote-as 65110
neighbor 198.51.100.10 ebgp-multihop 255
neighbor 198.51.100.10 update-source lo

ebgp-multihop 255: Allows the eBGP session to traverse multiple hops (required when the peer is not directly connected).
update-source lo: Uses the loopback interface as the source for BGP updates (ensures a stable source IP).

EVPN address family activation¶

address-family l2vpn evpn
neighbor 198.51.100.10 activate
exit-address-family

Activates the neighbor under the L2VPN EVPN address family, which enables EVPN route exchange.

BGP Unnumbered mode¶

```
bgp unnumbered-mode
neighbor tmm-spine1 remote-as external
exit-unnumbered-mode

address-family ipv4 unicast
bgp v4-unnumbered-mode
neighbor tmm-spine1 activate
exit-v4-unnumbered-mode
exit-address-family
```

bgp unnumbered-mode: Tells OcNOS to establish a BGP session using IPv6 link-local addresses on the named interface.
remote-as external: Indicates an eBGP session where the peer’s ASN is discovered dynamically.
v4-unnumbered-mode: Activates IPv4 route exchange over the IPv6 link-local BGP session.

How to upgrade OcNOS¶

The current release uses OcNOS version 6.6, upgraded from ZebOS/OcNOS 1.3.3. This section describes the key changes, new capabilities, and behavioral differences introduced by the upgrade.

What changed in OcNOS 6.6¶

Area	OcNOS 1.3.3 (previous)	OcNOS 6.6 (current)
Instance model	Multiple OcNOS instances, each in a separate Linux network namespace. One instance per VRF.	Single OcNOS instance with native VRF support. All VRFs run in one instance.
VRF support	Handled by running separate OcNOS processes in isolated namespaces. TMM managed namespace creation.	Native VRF support within OcNOS. TMM creates VRF interfaces in the kernel, and OcNOS manages routing tables per VRF.
EVPN	Not supported	Supported. EVPN RT-2 and RT-5 route exchange for VXLAN overlay.
VXLAN	Not supported	Supported. VXLAN tunnel encapsulation and decapsulation with VNI-based segmentation.
BGP Unnumbered	Not supported	Supported. Uses IPv6 link-local addresses for neighbor discovery and session establishment.
BFD	Supported through the old ZebOS BFD stack	Supported through the `oamd` daemon. TMM BFD Proxy handles BFD packets.
Configuration delivery	ConfigMap-based BGP configuration	CRD-based configuration with Go template rendering and gRPC delivery

Upgrade impact¶

Architecture changes¶

Single instance replaces multiple instances. In OcNOS 1.3.3, each VRF ran in its own OcNOS instance inside a separate network namespace. In OcNOS 6.6, a single OcNOS instance handles all VRFs natively. This reduces resource usage and simplifies management.
VRF interface creation moves to TMM. TMM now creates VRF interfaces in the Linux kernel (instead of relying on namespace isolation) and binds tap interfaces to the correct VRF. TMM sends VRF state to OcNOS through tmrouted.
Configuration delivery moves to gRPC. The old ConfigMap-based method is replaced by CRD-based configuration with Go template rendering. The CNE controller renders templates and pushes CLI payloads to OcNOS through gRPC.

CRDs¶

The following CRDs support the OcNOS 6.6 architecture:

CRD	Purpose
`GlobalRoutingConfig`	Defines cluster-wide BGP configuration
`RoutingConfig`	Defines tenant-specific BGP and VXLAN/VRF configuration
`RoutingTemplate`	Contains Go templates for rendering OcNOS CLI
`VRF`	Creates tenant VRFs with native VRF support
`VXLAN`	Configures VXLAN tunnels for tenant traffic

Behavioral changes¶

Behavior	OcNOS 1.3.3	OcNOS 6.6
VRF isolation	Network namespace separation	Linux VRF interfaces with `strictIsolation` flag
BGP session source	Per-namespace interface	Configurable via `update-source` in the RoutingTemplate
Route installation	Per-namespace routing table	Per-VRF routing table within a single OcNOS instance
Configuration updates	Edit ConfigMap and restart	Delete CRD and recreate (immutable CRDs)

TMM restart behavior¶

When TMM restarts, existing VRF and tap interfaces may already be present in the namespace from the previous instance. OcNOS 6.6 handles this gracefully:

TMM re-creates VRF interfaces if they are missing.
TMM re-binds tap interfaces to the correct VRF.
OcNOS re-establishes BGP sessions and resends current routing state.
Tmrouted clears previously learned ARL and ARP entries and receives a fresh sync from OcNOS.

Upgrade procedure¶

The OcNOS upgrade from 1.3.3 to 6.6 is handled automatically as part of the BIG-IP Next for Kubernetes platform upgrade. You do not upgrade OcNOS independently.

Migration from ConfigMap to CRD-based configuration¶

If you previously used the ConfigMap-based method to configure BGP, you must migrate to the new CRD-based method after upgrading.

General migration steps:

Document your current configuration. Capture your existing BGP settings from the ConfigMap, including ASNs, neighbor IPs, address families, and authentication.
Create a RoutingTemplate. Translate your existing OcNOS CLI configuration into a Go template.
Create supporting CRDs. Create VRF and VXLAN resources if your deployment uses VRF isolation or VXLAN overlays.
Create a GlobalRoutingConfig. Define your global BGP settings using the values from your existing ConfigMap.
Create RoutingConfig resources. If you have tenant-specific routing, create one RoutingConfig per tenant.
Verify. Check CRD status and confirm that BGP sessions are established and routes are exchanging correctly.
Remove the old ConfigMap. After you confirm that the CRD-based configuration is working, remove the old ConfigMap.