litellm-fips

packaged by Chainguard

Chainguard Container for litellm-fips

LiteLLM is a unified interface to call 100+ LLMs using the OpenAI format, providing a proxy server for multiple LLM providers.

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/litellm-fips:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

The litellm-fips image is a drop-in for the upstream litellm-non_root image, with the same configuration and entrypoint. It runs as the non-root user nonroot (UID 65532) rather than upstream's UID 65534, so files you mount, such as configuration or certificates, must be readable by UID 65532. In the supported configuration described below, all cryptography goes through the system OpenSSL and its FIPS 140-3 validated provider, which changes the following:

• Proxy user passwords are hashed with PBKDF2-HMAC-SHA256 instead of scrypt, and must be at least 8 bytes.
• Secrets stored in the database, such as credentials, are always encrypted with AES-256-GCM. general_settings.encryption_algorithm no longer selects the algorithm.
• The database must use TLS client-certificate authentication. A password DATABASE_URL still works, but Prisma then authenticates with SCRAM-SHA-256 outside the validated module.

MCP outbound OAuth credentials, MCP per-user environment variables, creating or updating MCP servers stored in the database, the CloudZero, FOCUS, Vantage, and Mavvrik cost exports, Bedrock audio transcription, --run_granian, USE_DDTRACE, LITELLM_ENABLE_PYROSCOPE, and the langfuse 2.x callback rely on non-approved cryptography and fail when enabled.

Two features make outbound calls by default. Deployments without internet access should turn both off:

• Breached-password screening calls api.pwnedpasswords.com when a password is set and at login. Set general_settings.password_policy_check_breached_passwords: false.
• At startup, the proxy fetches its model cost map from GitHub, retrying and logging warnings if the fetch fails. Set the LITELLM_LOCAL_MODEL_COST_MAP=True environment variable to use the map bundled in the image.

FIPS Support

The litellm-fips image ships a validated redistribution of OpenSSL's FIPS provider module. The proxy's Python runtime uses OpenSSL 4.0, and Prisma's query engine, which makes the database connections, uses OpenSSL 3.5 because Prisma publishes no engine for OpenSSL 4. Both load the same FIPS provider module. For more on FIPS support in Chainguard Containers, consult the guide on FIPS-enabled Chainguard Containers on Chainguard Academy.

To confirm that the image runs in FIPS mode, check the mode reported by its Python runtime:

docker run --rm --entrypoint python3 cgr.dev/ORGANIZATION/litellm-fips:latest \
  -c "from _hashlib import get_fips_mode; print(get_fips_mode())"

1

Moving from the non-FIPS litellm image is one-way. Re-encrypt stored secrets on the non-FIPS image first: set general_settings.encryption_algorithm: aes-256-gcm, run migrate_encryption, and confirm check_encryption reports residual_legacy: 0. Users with scrypt passwords need a password reset.

Getting Started

This example runs the proxy against a PostgreSQL container that accepts only certificate authentication. It uses OpenSSL 3.0 or later and a Linux host.

Create a self-signed certificate for the database and one for the litellm database role. Each side trusts the other's certificate, and Prisma reads the client identity from a PKCS12 bundle built without the legacy KDF, which the FIPS provider rejects:

mkdir -p pg-server pg-client
openssl req -x509 -newkey rsa:2048 -nodes -days 365 -subj /CN=litellm-db \
  -addext subjectAltName=DNS:litellm-db -keyout pg-server/server.key -out pg-server/server.crt
openssl req -x509 -newkey rsa:2048 -nodes -days 365 -subj /CN=litellm \
  -keyout pg-client/client.key -out pg-client/client.crt
openssl pkcs12 -export -nomac -certpbe AES-256-CBC -keypbe AES-256-CBC -passout pass:change-me \
  -inkey pg-client/client.key -in pg-client/client.crt -out pg-client/client.p12
cp pg-client/client.crt pg-server/ && cp pg-server/server.crt pg-client/
printf 'local all all trust\nhostssl all all all cert\n' > pg-server/pg_hba.conf
chmod 600 pg-server/server.key pg-client/client.key pg-client/client.p12
sudo chown -R 70:70 pg-server && sudo chown -R 65532:65532 pg-client

Start PostgreSQL with TLS and the certificate-only pg_hba.conf:

docker network create litellm
docker run -d --name litellm-db --network litellm \
  -e POSTGRES_USER=litellm -e POSTGRES_PASSWORD=unused -v "$PWD/pg-server:/certs:ro" \
  cgr.dev/ORGANIZATION/postgres:latest \
  -c ssl=on -c ssl_cert_file=/certs/server.crt -c ssl_key_file=/certs/server.key \
  -c ssl_ca_file=/certs/client.crt -c hba_file=/certs/pg_hba.conf

Start the proxy. Prisma reads sslcert as the database's certificate and the client identity from sslidentity, while the proxy's libpq-based startup checks read PGSSLCERT and PGSSLKEY:

docker run -d --name litellm --network litellm -p 4000:4000 \
  -v "$PWD/pg-client:/certs:ro" -e LITELLM_MASTER_KEY=sk-change-me \
  -e DATABASE_URL="postgresql://litellm@litellm-db:5432/litellm?sslmode=require&sslaccept=strict&sslcert=/certs/server.crt&sslidentity=/certs/client.p12&sslpassword=change-me" \
  -e PGSSLCERT=/certs/client.crt -e PGSSLKEY=/certs/client.key \
  cgr.dev/ORGANIZATION/litellm-fips:latest

After the database migrations finish, about a minute, the readiness endpoint reports the database as connected:

curl -s http://localhost:4000/health/readiness | jq .db

"connected"

The admin UI is available at http://localhost:4000/ui. Models are configured as for the litellm image.

Deploy with Helm

The upstream litellm-helm chart runs the image with the same certificate authentication. This example assumes the database is reachable in the cluster as litellm-db and uses the certificates from the first step, without the chown line, which only the Docker example needs.

Store the client certificates in a Secret, and create the credentials Secret the chart reads even though the URL carries no password:

kubectl create secret generic litellm-db-certs --from-file=pg-client/
kubectl create secret generic litellm-db --from-literal=username=litellm --from-literal=password=unused

Save the following as values.yaml. The fsGroup lets the proxy read the mounted certificates, and the Helm hook runs the database migrations before the proxy starts:

image:
  repository: cgr.dev/ORGANIZATION/litellm-fips
  tag: latest
masterkey: sk-change-me
podSecurityContext:
  fsGroup: 65532
db:
  deployStandalone: false
  useExisting: true
  url: postgresql://litellm@litellm-db:5432/litellm?sslmode=require&sslaccept=strict&sslcert=/certs/server.crt&sslidentity=/certs/client.p12&sslpassword=change-me
  secret:
    name: litellm-db
    usernameKey: username
    passwordKey: password
volumes:
  - name: db-certs
    secret:
      secretName: litellm-db-certs
      defaultMode: 0440
volumeMounts:
  - name: db-certs
    mountPath: /certs
    readOnly: true
envVars:
  PGSSLCERT: /certs/client.crt
  PGSSLKEY: /certs/client.key
migrationJob:
  hooks:
    helm:
      enabled: true

Install the chart, then forward the proxy's port:

helm install litellm oci://ghcr.io/berriai/litellm-helm -f values.yaml --wait --timeout 10m
kubectl port-forward svc/litellm 4000:4000

In a second terminal, check readiness:

curl -s http://localhost:4000/health/readiness | jq .db

"connected"

Documentation and Resources

• LiteLLM Proxy documentation
• PostgreSQL certificate authentication
• Chainguard FIPS container images

What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with Wolfi, our minimal Linux undistro.

All other Chainguard Containers are built with Chainguard OS, Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a development, or -dev, variant.

In all other cases, including Chainguard Containers tagged as :latest or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the -dev container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the -dev variant into a more minimal production image.

Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use Custom Assembly to add packages, either through the Console, chainctl, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in your Organization's list of images, and click on the Customize image button at the top of the page.

Learn More

Refer to our Chainguard Containers documentation on Chainguard Academy. Chainguard also offers VMs and Libraries — contact us for access.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.