Search. Pull. Build.

llm-d Router components — the Endpoint Picker (EPP) ext-proc endpoint selector, the prefill/decode disaggregation sidecar, and the request coordinator — for cache- and load-aware routing of LLM inference traffic.

FIPS-enabled llm-d Router components — the Endpoint Picker (EPP), the prefill/decode disaggregation sidecar, and the request coordinator — built with the go-msft toolchain so stdlib crypto routes through the FIPS-validated OpenSSL provider.

LMCache is an LLM serving engine extension that stores and reuses KV caches across requests to reduce time-to-first-token (TTFT) and increase throughput. It integrates with vLLM to provide GPU-accelerated inference with shared KV cache management.

Kubernetes operator for the vLLM Production Stack, vLLM's reference system for K8S-native cluster-wide deployment, that manages LoRA adapters, vLLM runtimes, routers and cache servers through CRDs.

FIPS-enabled Kubernetes operator for the vLLM Production Stack, vLLM's reference system for K8S-native cluster-wide deployment, that manages LoRA adapters, vLLM runtimes, routers and cache servers through CRDs.

Local Path Provisioner provides a way for the Kubernetes users to utilize the local storage in each node.

Minimal local-path-provisioner-fips image.