F5 EPP Custom Resource Reference

The F5EPP Custom Resource (CR) configures and manages an Endpoint Picker (EPP) instance for a target Kubernetes InferencePool.

Minimal Configuration

Only two parameters are strictly required to instantiate an F5EPP resource:

• spec.poolRef.name

• spec.engine

All other parameters are optional and override the automatic controller-generated defaults.

---

Field Specifications

metadata

Configures standard Kubernetes object metadata for the F5EPP resource.

Parameter	Type	Required	Description
name	String	Yes	Unique name of the F5EPP resource within the namespace.
namespace	String	Yes	Namespace for the F5EPP CR. Must match the target InferencePool namespace. Must also be included in the CNEInstance resource’s eppNamespaces list (unless set to "All").

---

spec

Configures the operational parameters, backend engine properties, and deployment configuration for the EPP instance.

Parameter	Type	Default	Description
poolRef	Object	Required	Defines the 1:1 binding to the target InferencePool. Immutable. See spec.poolRef.
engine	Enum	Required	Defines the underlying inference server backend: vllm, dynamo, or sglang. Determines metric collection, pod scoring algorithms, and KV-cache tracking.
mode	Enum	aggregated	Defines pool deployment topology: • aggregated: Every pod processes requests end-to-end. • disaggregated: Requests are routed across two distinct pod groups (prefill vs. decode generation).
blockSize	Integer	16	KV cache block size (in tokens) configured on the backend model servers (e.g., --block-size in vLLM). Used for prefix cache matching.
natsURL	String (URL)	""	NATS message bus endpoint for Dynamo cache events (e.g., nats://<service>.<namespace>.svc.cluster.local:4222). Required when engine: dynamo; forbidden for all other engines. Max length: 2048 characters.
tokenizer	Object	{}	Configures explicit tokenizer loading. If unconfigured, the picker auto-discovers the model name from engine metrics. See spec.tokenizer.
server	Object	{}	Overrides default network listener and pod polling intervals. See spec.server.
replicas	Integer	1	Number of EPP pods to deploy (Range: 1–2). EPP instances operate in an Active/Standby leader-election model.
podTemplate	Object	{}	Strategic merge patch applied to the generated EPP deployment pod template (e.g., node affinity, tolerations, resource limits, sidecars). See spec.podTemplate for restricted fields.
eppConfig	Object	{}	Overrides generated EPP picker configuration logic. See spec.eppConfig.
fluentBit	Object	{}	Optional fluent-bit logging sidecar. When enabled, the EPP’s logs are shipped to stdout and/or fluentd. Omitted or disabled, the EPP pod runs with no fluentbit sidecar. See spec.fluentBit.

spec.poolRef

Defines the explicit Kubernetes target resource for the EPP binding.

Parameter	Unit	Description	Default
poolRef.name	string	Name of the InferencePool this EPP serves. Must exist in the same namespace as this resource. Required. DNS-1123 subdomain (dots allowed). Max length: 253.	—
poolRef.group	string	API group of the referenced InferencePool. Only inference.networking.k8s.io is accepted.	inference.networking.k8s.io
poolRef.kind	string	Kind of the referenced resource. Only InferencePool is accepted.	InferencePool

poolRef is immutable: every rendered object is named from the pool, and cleanup derives those names from this field. To serve a different pool, delete this resource and create one for that pool.

spec.tokenizer

Configures custom tokenizer behavior when automatic metric-based discovery is insufficient (e.g., when model aliases are used or files are stored locally).

Parameter	Unit	Description	Default
tokenizer.name	string	Model name whose tokenizer to load, usually the HuggingFace repository name. Setting it is what turns this block into a pin. Leaving it out means the picker takes the model name from the engine’s metrics; source and path are then rejected. Max length: 253.	"" (auto-discover)
tokenizer.source	enum	Where the tokenizer files come from. huggingface downloads them from HuggingFace Hub. local reads them from a path already inside the pod, which you have to mount yourself through spec.podTemplate.	—
tokenizer.path	string	Path to the tokenizer files inside the picker’s container. Required when source: local; ignored when source: huggingface. Max length: 1024.	""
tokenizer.cacheClaim	string	Name of a PVC you have already created in this namespace, mounted where the picker caches downloaded tokenizer files. Without it that cache is a temporary directory thrown away on every pod restart. The controller only references the claim; it never creates, resizes or deletes storage. Max length: 253.	""
tokenizer.hfTokenSecret	object	Secret in this namespace holding a HuggingFace access token, passed to the picker as HF_TOKEN. Needed only when the tokenizer lives in a gated or private HuggingFace repository. See table below.	{}

spec.tokenizer.hfTokenSecret

Parameter	Unit	Description	Default
hfTokenSecret.name	string	Name of the Secret, in the same namespace as this resource. Required per entry. Max length: 253.	—
hfTokenSecret.key	string	Key inside that Secret holding the token. Max length: 253.	token

Note: Getting the HuggingFace token wrong fails quietly — without a usable token every tokenizer download is refused with 401, so the picker cannot compare prompts at all, while it keeps serving traffic and reporting healthy. Naming a Secret that does not exist yet does not block the pod: it starts without a token instead of failing to start.

spec.server

Configures network scrape target configurations and listener parameters.

Parameter	Unit	Description	Default
server.modelServerMetricsPort	integer	Port on each model-server pod that the picker scrapes for metrics. Range: 1–65535.	vllm: 8000, dynamo: 9090, sglang: 30000
server.refreshMetricsInterval	string (Go duration)	How often the picker re-reads each model server’s metrics, e.g. 50ms, 1s.	50ms
server.secureServing	boolean	Whether to require TLS on the gRPC listener that BNK calls. Must be false. The field is kept in the schema for when the proxy can use TLS.	false
server.metricsEndpointAuth	boolean	Whether the picker requires a token on its own /metrics endpoint. Must be false. The field is kept in the schema for when the picker’s permissions can carry it.	false

spec.podTemplate

Defines custom parameters injected into the generated Deployment specification.

Reserved (Forbidden) Fields

The controller explicitly rejects modifications to the following fields to guarantee system stability:

• spec.containers[*].args

• spec.containers[*].command

• spec.containers[*].ports

• spec.containers[*].readinessProbe

• spec.containers[*].lifecycle

• spec.serviceAccountName

• The epp-config volume and volume mount path.

• Reserved Environment Variables: EPP_POOL_NAME, EPP_POOL_NAMESPACE, EPP_CONFIG, EPP_CONFIG_FILE

NOTE: The livenessProbe configuration is not reserved and can be customized to tune container failure thresholds.

spec.eppConfig

Provides overrides for internal picker execution profiles. config and configMapRef are mutually exclusive.

Parameter	Unit	Description	Default
eppConfig.config	object	Settings merged on top of the controller-generated configuration. Plugins merge by name, profiles merge by name and their entries by pluginRef. Setting one parameter, or one weight, leaves everything else generated and still on the upgrade path — send only the keys you are changing. See sub-table below.	{}
eppConfig.configMapRef	object	Names a ConfigMap you supply carrying the picker’s entire configuration in two keys: pool.yaml (decision steps and scoring profiles) and epp.yaml (the picker process’s own settings). When set, the controller generates neither key and mounts your ConfigMap as-is. You also own the rollout — the picker reads its configuration once at startup, so editing this ConfigMap changes nothing until you kubectl rollout restart deployment/<pool>-epp.	{}

spec.eppConfig.config

Parameter	Unit	Description	Default
config.plugins	list of objects	Plugin entries merged over the generated ones by name (a plugin’s name defaults to its type). Setting parameters: {"$patch": "delete"} on an entry removes that plugin. Deleting and declaring the same plugin in a single override is rejected.	[]
config.schedulingProfiles	list of objects	Scoring profiles merged over the generated ones by name. Within a profile, entries merge by pluginRef — changing how the generated scores are blended means sending just the profile name and the entries whose weight changed.	[]
config.featureGates	list of strings	Experimental picker features to switch on, by name. Replaces the generated list outright rather than adding to it, so repeat any generated entry you still want.	[]
config.flowControl	object	Queue capacity and priority bands, honoured only when the flowControl feature gate is switched on. Replaces the generated block outright rather than merging key by key.	{}

Note: Direct parameter settings for eppConfig.config.data and eppConfig.config.saturationDetector are invalid and will be rejected.

• Configure data extraction settings via config.plugins.

• Configure saturation thresholds through the utilization-detector plugin parameters (queueDepthThreshold, kvCacheUtilThreshold, metricsStalenessThreshold, headroom).

spec.fluentBit

Configures the optional fluent-bit logging sidecar for the EPP pod. When enabled, fluent-bit ships the EPP’s logs to stdout and/or forwards them to a central fluentd. Omitted or disabled, the EPP pod runs with no fluent-bit sidecar.

Parameter	Unit	Description	Default
fluentBit.enabled	boolean	Whether to add the fluent-bit sidecar. Required when this block is set.	—
fluentBit.image	string	Overrides the fluent-bit image. Leave empty to use the platform-resolved image.	""
fluentBit.resources	object	Resource requests and limits for the sidecar container, each a map of quantity strings (e.g. cpu: "1", memory: "256Mi").	{}
fluentBit.fluentbit	object	Tunes the fluent-bit engine (log level, flush interval, input). See table below.	{}
fluentBit.output	object	Selects where fluent-bit ships logs (stdout and/or fluentd). See table below.	{}

spec.fluentBit.fluentbit

Parameter	Unit	Description	Default
fluentbit.logLevel	string	fluent-bit’s own log level: trace, debug, info, warn, or error.	info
fluentbit.flushInterval	string	Flush interval, in seconds.	5
fluentbit.input	object	Tunes the named-pipe input buffer and poll interval. See table below.	{}

spec.fluentBit.fluentbit.input

Parameter	Unit	Description	Default
input.bufSize	string	Buffer size for the input plugin, e.g. 32k.	32k
input.intervalSec	integer	Poll interval, in seconds.	1
input.intervalNsec	integer	Poll interval, in nanoseconds (added to intervalSec).	0

spec.fluentBit.output

Parameter	Unit	Description	Default
output.stdout	boolean	Whether to keep the stdout output, so kubectl logs continues to work.	true
output.fluentd	object	Forwards logs to a central fluentd. See table below.	{}

spec.fluentBit.output.fluentd

Parameter	Unit	Description	Default
fluentd.enabled	boolean	Whether to forward logs to a central fluentd. Required when this block is set. Forwarding requires mTLS; enabling it without a valid tls block means logs are not delivered to fluentd, while the EPP itself keeps running.	—
fluentd.host	string	Hostname or address of the central fluentd.	""
fluentd.port	integer	Port of the central fluentd forward input.	54321
fluentd.timeAsInteger	boolean	Whether to emit event timestamps as integers, as some fluentd versions require.	true
fluentd.tls	object	mTLS configuration for the fluentd forward output. See table below.	{}

spec.fluentBit.output.fluentd.tls

Parameter	Unit	Description	Default
tls.enabled	boolean	Whether to use mTLS on the connection to fluentd. Required when this block is set.	—
tls.verify	boolean	Whether fluent-bit verifies the fluentd server certificate.	true
tls.debug	integer	fluent-bit TLS debug verbosity. Range: 0–4.	0
tls.secretName	string	Name of a pre-created Secret in this namespace holding the mTLS client certificate. Leave empty to use the platform-provided client certificate. Max length: 253.	""

Note: EPP fluent-bit to fluentd TLS communication is not supported when f5-cert-manager is being used.

---

Resource Status

The controller populates status with runtime metrics, managed resource references, and state validations.

status.conditions

Conditions follow a strict dependency sequence; subsequent conditions require preceding conditions to evaluate to True.

Condition	Description
Accepted	Resource specification has passed structural and rule validation.
ResolvedRefs	The referenced InferencePool exists, is accessible, and has no existing F5EPP bindings.
Programmed	Controller has provisioned all child resources (ConfigMap, Deployment, Service, ServiceAccount, Role, RoleBinding).
Available	Minimum required EPP pods are running (Ready) and passing gRPC health probes.

status.resolved

Displays active runtime configurations applied to the operational EPP pod.

Parameter	Type	Description
engine	String	Active inference engine (vllm, dynamo, or sglang).
mode	String	Active pool mode (aggregated or disaggregated).
model	String	Active model identifier. Empty if dynamically discovered.
modelSource	String	Origin of model identification (Spec or Discovered).
configSource	String	Origin of runtime EPP configuration (Generated, Merged, or OperatorSupplied).
natsURL	String	Resolved NATS endpoint URL (Dynamo engine only).
blockSize	Integer	Active KV cache block token sizing.

status.refs

Contains generated resource identifiers managed by the controller.

Parameter	Type	Description
configMap	String	Name of the generated ConfigMap (<pool>-epp-config).
deployment	String	Name of the generated Deployment (<pool>-epp).
service	String	Name of the generated Service (<pool>-epp).

NOTE: EPP deployment state metrics are exposed via status.replicas and status.readyReplicas.

Validation Rules & Dependencies

• Engine and Transport Requirements: spec.engine: dynamo requires spec.natsURL to be populated. The natsURL parameter is rejected for all non-Dynamo engines.

• Tokenizer Parameter Gates: spec.tokenizer.source and spec.tokenizer.path require spec.tokenizer.name to be populated.

• Local Tokenizer Path: When spec.tokenizer.source: local is specified, spec.tokenizer.path must be defined.

• Binding Cardinality: EPP-to-InferencePool binding is strictly 1:1. Duplicate bindings to an InferencePool cause validation failures on ResolvedRefs.

• Configuration Mutex: Configuring both spec.eppConfig.config and spec.eppConfig.configMapRef simultaneously will cause a validation rejection.

• Non-empty Configuration Overrides: If spec.eppConfig.config is set, it must contain at least one definition block (plugins, schedulingProfiles, featureGates, or flowControl).

• Failure Modes: EPP consulting failure behaviors are defined at the pool level via InferencePool.spec.endpointPickerRef.failureMode (Defaults to FailClose).

• Namespace Scoping: The controller resolves F5EPP instances only within namespaces listed in CNEInstance.spec.eppNamespaces.

• fluent-bit Sidecar: spec.fluentBit.enabled must be true to render the sidecar. If no fluent-bit image can be resolved (neither spec.fluentBit.image nor the platform-provided image), the EPP pod is created without the sidecar.

• fluentd mTLS: Forwarding to a central fluentd requires mTLS; enabling spec.fluentBit.output.fluentd without a valid tls configuration results in logs not being delivered. EPP fluent-bit to fluentd TLS communication is not supported when f5-cert-manager is being used.