llm-d-router-disagg-sidecar-fips

packaged by Chainguard

Chainguard Container for llm-d-router-disagg-sidecar-fips

FIPS-enabled llm-d Router components — the Endpoint Picker (EPP), the prefill/decode disaggregation sidecar, and the request coordinator — built with the go-msft toolchain so stdlib crypto routes through the FIPS-validated OpenSSL provider.

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/llm-d-router-disagg-sidecar-fips:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

These are the FIPS variants of the llm-d Router components — drop-in replacements for the standard images with identical entrypoints, ports, working directory, and configuration, published under the same names with a -fips suffix:

All three images run as the nonroot user 65532 and set SSL_CERT_FILE to the system CA bundle for outbound TLS. The coordinator ships the upstream default configuration at /config/coordinator/coordinator.yaml, and its entrypoint reads it. The disaggregation sidecar keeps the upstream working directory /home/nonroot. Switching from the upstream images or from the standard Chainguard images to these FIPS variants should not require changes to your manifests or Helm values beyond the image references.

FIPS Support

These images are built with the OpenSSL-backed Go FIPS toolchain (go-msft), so the Go standard library's crypto/* routines are serviced by the FIPS-validated OpenSSL provider rather than Go's native crypto:

• FIPS-validated OpenSSL provider — cryptographic operations run through the OpenSSL FIPS module.
• FIPS-approved algorithms — TLS and standard-library cryptography run through the OpenSSL FIPS provider. The one exception is the disaggregation sidecar's unkeyed BLAKE2s digest, which only picks a data-parallel rank and is not a security operation.
• FIPS mode enforcement — the binaries fail closed when the FIPS provider cannot be loaded.

For more on FIPS support in Chainguard container images, see the guide on FIPS-enabled Chainguard Containers.

Getting Started

The three components are independent images:

cgr.dev/ORGANIZATION/llm-d-router-endpoint-picker-fips:latest
cgr.dev/ORGANIZATION/llm-d-router-disagg-sidecar-fips:latest
cgr.dev/ORGANIZATION/llm-d-router-coordinator-fips:latest

The Endpoint Picker plugs into a data-plane proxy (Envoy or agentgateway) over the Envoy ext_proc gRPC protocol and watches Gateway API Inference Extension resources to make routing decisions. The disaggregation sidecar fronts a vLLM model server in prefill/decode pods and coordinates KV-cache transfer. The coordinator exposes an OpenAI-compatible HTTP endpoint that orchestrates the encode/prefill/decode pipeline.

The coordinator and sidecar start standalone; the Endpoint Picker needs a Kubernetes cluster. For example, the coordinator serves its readiness endpoint over TLS (a self-signed certificate by default) using the baked configuration:

docker run -d -p 8080:8080 cgr.dev/ORGANIZATION/llm-d-router-coordinator-fips:latest
curl -skf --retry 20 --retry-all-errors https://localhost:8080/readyz

The Endpoint Picker is deployed with the upstream llm-d-router-standalone Helm chart (oci://ghcr.io/llm-d/charts/llm-d-router-standalone). Install the Gateway API Inference Extension CRDs first, then install the chart, pointing the image at the FIPS repository:

# values.yaml
router:
  epp:
    image:
      registry: cgr.dev/ORGANIZATION
      repository: llm-d-router-endpoint-picker-fips
      tag: 0.11.0
  # Required: label selector for your model-server pods. The chart refuses to
  # render without it.
  modelServers:
    matchLabels:
      app: my-vllm

helm install llm-d-router oci://ghcr.io/llm-d/charts/llm-d-router-standalone \
  --version v0.11.0 \
  --namespace llm-d --create-namespace \
  --values values.yaml

Use the chart version that matches your image tag. The chart tag carries a leading v, while the image tag does not — image tag 0.11.0 corresponds to --version v0.11.0.

Configuration

The coordinator is configured through /config/coordinator/coordinator.yaml (server listen address, gateway address, and pipeline connectors); override it by mounting your own file and pointing --config at it. The disaggregation sidecar is configured with flags such as --port, --model-server-port, --kv-connector, and --secure-proxy. The Endpoint Picker is configured through the Helm chart values (pool selection, scheduler plugins, and the proxy type). Refer to the upstream documentation for the full option sets.

Documentation and Resources

• llm-d Router repository
• llm-d quickstart
• Helm chart reference
• Chainguard FIPS documentation

What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with Wolfi, our minimal Linux undistro.

All other Chainguard Containers are built with Chainguard OS, Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a development, or -dev, variant.

In all other cases, including Chainguard Containers tagged as :latest or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the -dev container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the -dev variant into a more minimal production image.

Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use Custom Assembly to add packages, either through the Console, chainctl, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in your Organization's list of images, and click on the Customize image button at the top of the page.

Learn More

Refer to our Chainguard Containers documentation on Chainguard Academy. Chainguard also offers VMs and Libraries — contact us for access.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.