dstack

dstack is an open-source orchestration layer built specifically for AI and ML workloads. It gives teams a single control plane to run dev environments, training jobs, and inference services across distributed compute resources, whether those live in AWS, GCP, Azure, Lambda, RunPod, a Kubernetes cluster, or bare-metal servers with SSH access.

The core problem it solves: AI infrastructure is fragmented. Teams end up writing custom glue code for each environment, or fighting general-purpose tools like Kubernetes or Slurm that weren't designed with GPU workloads in mind. dstack abstracts that away with ML-native primitives.

Key capabilities:

• Fleets manage pools of GPU instances with fine-grained controls over spot, on-demand, and reserved capacity, helping reduce effective GPU spend.
• Dev environments let researchers spin up GPU-backed workspaces without touching infrastructure config.
• Tasks support both single-node and distributed training runs across heterogeneous hardware.
• Services handle production inference deployments with the same configuration interface.
• Volumes provide persistent storage that follows workloads across environments.
• Multi-cloud and on-prem support covers NVIDIA, AMD, Tenstorrent, and TPU accelerators in a single workflow.

If you already run Kubernetes, dstack connects to existing clusters via kubeconfig and schedules workloads alongside cloud backends. Bare-metal servers with SSH access work the same way. You can mix cloud accounts, Kubernetes clusters, and on-prem nodes in one project.

For teams without existing cloud accounts, dstack Sky offers a hosted GPU marketplace with access to H100s, A100s, B200s, and other hardware at competitive rates.

Compared to Slurm, dstack is container-first and cloud-native, with built-in support for inference and development (not just batch jobs). Compared to raw Kubernetes, it's purpose-built for ML: you declare what you need in simple configuration, and dstack handles provisioning, scheduling, and cost optimization. It works with any open-source framework and your own Docker images, so there's no lock-in to a specific ML stack.