vllm-omni-fips

packaged by Chainguard

Chainguard Container for vllm-omni-fips

vLLM-Omni extends vLLM to serve omni-modality models, covering text, image, video, and audio, through an OpenAI-compatible API. This FIPS-validated variant provides OpenSSL FIPS 140-3 compliance.

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/vllm-omni-fips:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

Chainguard's vllm-omni-fips image is a FIPS variant of the Chainguard vllm-omni image. It is comparable to the upstream vllm/vllm-omni image. It runs vLLM-Omni 0.30.0 on vLLM 0.30.0 with CUDA 13.0 and Python 3.12, matching the vLLM version upstream builds on, and is available for amd64 and arm64.

The image differs from the upstream image in the following ways:

• Like upstream, the image has no default command, so you pass the command to run, such as vllm serve. The entrypoint first checks that the NVIDIA driver and CUDA forward compatibility are usable, then runs your command.
• vLLM-Omni is installed in a virtual environment at /opt/chainguard/vllm-omni, which is on the PATH. The working directory is /app, but the image does not include the vLLM-Omni source tree that upstream copies there.
• Python packages whose upstream wheels bundle their own cryptography or TLS libraries, such as cryptography, grpcio, pyzmq, and opencv-python-headless, are replaced with Chainguard builds that use the system OpenSSL. PyAV is built against the image's FFmpeg, and imageio-ffmpeg uses /usr/bin/ffmpeg instead of a bundled binary.
• vLLM's multimodal input hasher defaults to sha256 instead of blake3. To use blake3, pass --mm-hasher-algorithm blake3.
• nvcc from CUDA 13.0 does not support the default GCC 16, so the image routes it to GCC 15 for kernels compiled at runtime.
• The image runs as root, as upstream does, so the container can access GPU devices.
• The FIPS image adds openssl-config-fipshardened, which configures the system OpenSSL to use its FIPS provider. The packages that would otherwise bundle their own cryptography use the system OpenSSL, so their cryptography goes through the FIPS provider as well.

FIPS Support

The vllm-omni-fips Chainguard Container ships with a validated redistribution of OpenSSL's FIPS provider module. For more on FIPS support in Chainguard Containers, consult the guide on FIPS-enabled Chainguard Containers on Chainguard Academy.

The Run:ai Model Streamer loads models from object storage through prebuilt backends from upstream. The Google Cloud Storage backend statically links its own TLS stack rather than the system OpenSSL, so models loaded through it are outside this image's FIPS boundary. For a FIPS-covered transport, download the model with a FIPS-validated client and point vLLM at the local path.

Prerequisites

• An NVIDIA GPU with a driver that supports CUDA 13.0
• The NVIDIA Container Toolkit
• Docker with GPU support enabled

Getting Started

Start the OpenAI-compatible server with a small model to verify that the image works on your GPU:

docker run -d --name vllm-omni \
  --gpus all \
  --shm-size 1g \
  -p 8000:8000 \
  -v "$HOME/.cache:/root/.cache" \
  cgr.dev/ORGANIZATION/vllm-omni-fips:latest \
  vllm serve facebook/opt-125m \
  --gpu-memory-utilization 0.7 \
  --max-num-seqs 8 \
  --host 0.0.0.0 \
  --port 8000

The model is downloaded into $HOME/.cache on the first run. Once the server reports Application startup complete, check its health and list the served models:

curl http://localhost:8000/health
curl http://localhost:8000/v1/models

Then request a completion:

curl http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "facebook/opt-125m", "prompt": "Deep learning is", "max_tokens": 10, "temperature": 0.0}'

To serve an omni-modality model, add the --omni flag and choose a model that vLLM-Omni supports. For example, to serve Qwen2.5-Omni:

docker run -d --name vllm-omni-qwen \
  --gpus all \
  --shm-size 8g \
  -p 8000:8000 \
  -v "$HOME/.cache:/root/.cache" \
  cgr.dev/ORGANIZATION/vllm-omni-fips:latest \
  vllm serve Qwen/Qwen2.5-Omni-3B --omni \
  --host 0.0.0.0 \
  --port 8000

Omni-modality models need considerably more GPU memory than the small model above, and vLLM-Omni's default deploy config for Qwen2.5-Omni places its stages on two GPUs. Refer to the vLLM-Omni documentation for the list of supported models and their requirements.

To print the vLLM-Omni version, run:

docker run --rm --gpus all cgr.dev/ORGANIZATION/vllm-omni-fips:latest vllm-omni --omni --version

To confirm that the FIPS provider is active, compute an HMAC with a key that meets the FIPS minimum length of 112 bits:

docker run --rm --entrypoint python3 cgr.dev/ORGANIZATION/vllm-omni-fips:latest \
  -c "import hmac; print(hmac.digest(b'14charslongkey', b'', 'SHA256').hex())"

0db994567d50545ac5a44823f82aae06b1a21b99f8dd0a42b3d572b1af62f182

The same call with a key shorter than 112 bits, such as b'shortkey', fails with a ValueError, because the FIPS provider enforces the minimum key length.

Documentation and Resources

• vLLM-Omni documentation
• vLLM-Omni on GitHub
• vLLM documentation
• NVIDIA Container Toolkit installation guide
• FIPS-enabled Chainguard Containers on Chainguard Academy
• Getting Started with Chainguard Containers on Chainguard Academy

What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with Wolfi, our minimal Linux undistro.

All other Chainguard Containers are built with Chainguard OS, Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a development, or -dev, variant.

In all other cases, including Chainguard Containers tagged as :latest or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the -dev container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the -dev variant into a more minimal production image.

Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use Custom Assembly to add packages, either through the Console, chainctl, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in your Organization's list of images, and click on the Customize image button at the top of the page.

Learn More

Refer to our Chainguard Containers documentation on Chainguard Academy. Chainguard also offers VMs and Libraries — contact us for access.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.