Open Source Local Model Runners using Cargo
A curated collection of the best open source run and serve large language models locally or on your own infrastructure. using Cargo.
High-throughput, memory-efficient serving engine for LLMs
Stars
95,239Last commit
8 hours agoLicense
Apache-2.0
Inference and serving engine for large language models, built for speed and hardware efficiency with an OpenAI-compatible API and support for a wide range of open models.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
49,884Last commit
10 hours agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
Run AI models locally or connect to cloud, privately
Stars
45,049Last commit
8 hours agoLicense
Apache-2.0
Jan runs open-source AI models on your own hardware or connects to cloud providers like OpenAI, Anthropic, and Google, keeping your conversations off third-party servers.
Open Source Alternative to:
Openlane – Open-source, developer-first platform for automated compliance, risk management, and built-in Trust Center.
Bright Data