dask-kubernetes

packaged by Chainguard

Chainguard Container for dask-kubernetes

Native Kubernetes integration for Dask

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/dask-kubernetes:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

Chainguard's Dask Kubernetes image is comparable to the dask/dask-kubernetes-operator image maintained by Dask. It runs the same kopf-driven operator and reconciles the same DaskCluster, DaskWorkerGroup, and DaskJob CRDs, but the runtime is rebuilt on Chainguard OS and differs from upstream in a few ways that may matter when migrating:

• Non-root by default. The container runs as user dask (uid 1000, gid 1000); the upstream image runs as root. If you mount volumes or write files at runtime, ensure they are writable by uid 1000 or set securityContext.runAsUser accordingly.
• Python 3.10, not 3.8.13. Upstream pins an end-of-life'd Python 3.8.13; the Chainguard image ships Python 3.10. Any custom wheels installed alongside the operator must be compatible with 3.10.
• No shell wrapper. Upstream invokes the operator via ["/bin/sh", "-c", "kopf run -m dask_kubernetes.operator --verbose --all-namespaces"]. The Chainguard image has no shell — the entrypoint is kopf run -m dask_kubernetes.operator.controller directly (the same controller code path) and the default args add a liveness HTTP endpoint at http://0.0.0.0:8080/healthz so Kubernetes can probe the operator without exec-ing into the container.
• Minimal runtime. No shell, no package manager, no build toolchain. kubectl exec ... -- /bin/bash will not work; use the -dev variant for debugging.
• Dedicated venv on PATH / PYTHONPATH. The operator and its dependencies live under /usr/share/dask-kubernetes/ (PATH prefixed with /usr/share/dask-kubernetes/bin, PYTHONPATH set to the venv's site-packages). Upstream installs into the system Python — plugins copied to /usr/local/lib/python*/site-packages will not be picked up; install them into /usr/share/dask-kubernetes/lib/python3.10/site-packages instead.

See upstream's Dask Kubernetes documentation for operator configuration and CRD reference.

Getting Started

Install the operator

helm repo add dask https://helm.dask.org
helm install dask-k8s-operator dask/dask-kubernetes-operator \
  --set image.name=cgr.dev/ORGANIZATION/dask-kubernetes \
  --set image.tag=latest \
  --create-namespace -n dask-operator

Verify its running: 
`kubectl get pods -n dask-operator`

To get started with a dask cluster and scale it run this python snippet:

from dask_kubernetes.operator import KubeCluster cluster = KubeCluster( name="my-dask-cluster", image="ghcr.io/dask/dask:latest" ) cluster.scale(10)

You can see the dask worker nodes created above

kubectl get daskcluster -n default kubectl get pods -l dask.org/cluster-name=my-dask-cluster

## Documentation and Resources

* [Official Dask Kubernetes documentation](https://docs.dask.org/en/latest/deploying-kubernetes.html#dask-kubernetes-operator)
* [Default `values.yaml` file for Dask Kubernetes](https://github.com/dask/dask-kubernetes/blob/main/dask_kubernetes/operator/deployment/helm/dask-kubernetes-operator/values.yaml)

## What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with [Wolfi](https://edu.chainguard.dev/open-source/wolfi/overview?utm_source=readmes), our minimal Linux _[undistro](https://edu.chainguard.dev/open-source/wolfi/overview/#why-undistro)_.

All other Chainguard Containers are built with [Chainguard OS](https://edu.chainguard.dev/chainguard/chainguard-os/overview/?utm_source=readmes), Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

* Minimal design, without unnecessary software bloat
* Daily builds to ensure container images are up-to-date with available security patches
* [High quality build-time SBOMs](https://edu.chainguard.dev/chainguard/chainguard-images/working-with-images/retrieve-image-sboms/?utm_source=readmes) attesting to the provenance of all artifacts within the image
* [Verifiable signatures](https://edu.chainguard.dev/chainguard/chainguard-images/working-with-images/retrieve-image-sboms/) provided by [Sigstore](https://edu.chainguard.dev/open-source/sigstore/cosign/an-introduction-to-cosign/?utm_source=readmes)
* Reproducible builds with Cosign and apko ([read more about reproducibility](https://www.chainguard.dev/unchained/reproducing-chainguards-reproducible-image-builds?utm_source=readmes))

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with [a *development*, or `-dev`, variant](https://edu.chainguard.dev/chainguard/chainguard-images/about/differences-development-production/).

In all other cases, including Chainguard Containers tagged as `:latest` or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the `-dev` container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the `-dev` variant into a more minimal production image.

## Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use [Custom Assembly](https://edu.chainguard.dev/chainguard/chainguard-images/features/ca-docs/custom-assembly/) to add packages, either through the Console, `chainctl`, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in [your Organization's list of images](https://console.chainguard.dev/images/organization), and click on the **Customize image** button at the top of the page.

## Learn More

Refer to our Chainguard Containers documentation on [Chainguard Academy](https://edu.chainguard.dev/?utm_source=readmes). Chainguard also offers [VMs](https://www.chainguard.dev/vms?utm_source=readmes) and [Libraries](https://www.chainguard.dev/libraries?utm_source=readmes) — [contact us](https://www.chainguard.dev/contact?utm_source=readmes) for access.
 

## Trademarks

_This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies._