LocalAI

LocalAI is a self-hosted runtime that lets you run virtually any AI workload on hardware you control. Text generation, vision, speech recognition, text-to-speech, image and video generation, embeddings, reranking, and autonomous agents all run behind a single OpenAI-compatible API. If you're already using OpenAI or Anthropic APIs, switching the endpoint is often all it takes.

The core design is deliberately lean. Backends aren't bundled upfront. They're pulled on demand when a model needs them, each one wrapping a best-in-class engine like llama.cpp, vLLM, SGLang, MLX, or whisper.cpp as an isolated service. You can install, update, or remove individual backends without touching the rest of the stack. Hardware mixing is first-class: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, and Jetson all work, and you can route across them in a single cluster.

For cases where existing engines are too heavy or too closed, the LocalAI team builds its own:

• parakeet.cpp for streaming multilingual speech recognition
• vibevoice.cpp for long-form TTS and ASR
• voice-detect.cpp for speaker recognition and anti-spoofing
• face-detect.cpp for vision-based identity analysis
• privacy-filter.cpp for native PII detection and redaction
• apex-quant for MoE-aware GGUF quantization

It scales from a CPU-only laptop to a distributed GPU cluster without changing how you interact with it. A single workstation setup can grow into a team server with API keys, roles, quotas, and usage tracking, then further into a multi-worker cluster with model routing and device-spanning inference. rarely cover this range in one package.