Open Source Local Model Runners using C++
A curated collection of the best open source run and serve large language models locally or on your own infrastructure. using C++.
Run open-source LLMs locally on your own machine
Stars
181,894Last commit
8 hours agoLicense
MIT
Runs open-source language models locally with a simple setup, plus optional cloud access for larger models when local hardware isn't enough.
Open Source Alternative to:
Run LLMs locally with minimal setup, maximum hardware support
Stars
129,836Last commit
2 hours agoLicense
MIT
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
Open Source Alternative to:
High-throughput, memory-efficient serving engine for LLMs
Stars
92,916Last commit
2 hours agoLicense
Apache-2.0
Inference and serving engine for large language models, built for speed and hardware efficiency with an OpenAI-compatible API and support for a wide range of open models.
Open Source Alternative to:
Run open-source AI models privately on your own device
Stars
77,382Last commit
1 year agoLicense
MIT
GPT4All runs open-source language models locally on Windows, macOS, and Linux with no cloud dependency, keeping your data on your machine.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
49,315Last commit
4 hours agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
Capture – Capture screenshots, generate PDFs, scrape content, extract metadata, and automate browsers with one API.
Docmost