Open Source Local Model Runners using Cargo
A curated collection of the best open source run and serve large language models locally or on your own infrastructure. using Cargo.
High-throughput, memory-efficient serving engine for LLMs
Stars
93,122Last commit
7 hours agoLicense
Apache-2.0
Inference and serving engine for large language models, built for speed and hardware efficiency with an OpenAI-compatible API and support for a wide range of open models.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
49,371Last commit
7 hours agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
Run AI models locally or connect to cloud, privately
Stars
44,776Last commit
2 days agoLicense
Apache-2.0
Jan runs open-source AI models on your own hardware or connects to cloud providers like OpenAI, Anthropic, and Google, keeping your conversations off third-party servers.
Open Source Alternative to:
Sent – A unified API for sending messages across SMS, RCS, and apps like WhatsApp.
Sent