dstack

dstack is an open-source orchestration layer built specifically for AI and ML workloads. It gives teams a single control plane to run dev environments, training jobs, and inference services across distributed compute resources, whether those live in AWS, GCP, Azure, Lambda, RunPod, a Kubernetes cluster, or bare-metal servers with SSH access.

The core problem it solves: AI infrastructure is fragmented. Teams end up writing custom glue code for each environment, or fighting general-purpose tools like Kubernetes or Slurm that weren't designed with GPU workloads in mind. dstack abstracts that away with ML-native primitives.

Key capabilities:

• Fleets manage pools of GPU instances with fine-grained controls over spot, on-demand, and reserved capacity, helping reduce effective GPU spend.
• Dev environments let researchers spin up GPU-backed workspaces without touching infrastructure config.
• Tasks support both single-node and distributed training runs across heterogeneous hardware.
• Services handle production inference deployments with the same configuration interface.
• Volumes provide persistent storage that follows workloads across environments.
• Multi-cloud and on-prem support covers NVIDIA, AMD, Tenstorrent, and TPU accelerators in a single workflow.