Open Source Local Model Runners using Python
A curated collection of the best open source run and serve large language models locally or on your own infrastructure. using Python.
Run LLMs locally with minimal setup, maximum hardware support
Stars
124,707Last commit
6 hours agoLicense
MIT
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
Open Source Alternative to:
Run open-source AI models privately on your own device
Stars
77,404Last commit
1 year agoLicense
MIT
GPT4All runs open-source language models locally on Windows, macOS, and Linux with no cloud dependency, keeping your data on your machine.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
48,580Last commit
8 hours agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
Run AI models locally or connect to cloud, privately
Stars
44,052Last commit
10 hours agoLicense
Apache-2.0
Jan runs open-source AI models on your own hardware or connects to cloud providers like OpenAI, Anthropic, and Google, keeping your conversations off third-party servers.
Open Source Alternative to:
CodeRabbit – The leading AI Code Review platform. Ship better quality code in 50% less time, with 90% fewer bugs.
Stellar Hosted