Open Source Machine Learning Infrastructure Tools
A curated collection of the best open source infrastructure components specifically designed for AI/ML workloads..
Run open-source LLMs locally on your own machine
Stars
185,701Last commit
11 hours agoLicense
MIT
Runs open-source language models locally with a simple setup, plus optional cloud access for larger models when local hardware isn't enough.
Open Source Alternative to:
Run LLMs locally with minimal setup, maximum hardware support
Stars
133,181Last commit
10 hours agoLicense
MIT
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
Open Source Alternative to:
High-throughput, memory-efficient serving engine for LLMs
Stars
Stars growing over the last 30 days96,158Last commit
10 hours agoLicense
Apache-2.0
Inference and serving engine for large language models, built for speed and hardware efficiency with an OpenAI-compatible API and support for a wide range of open models.
Open Source Alternative to:
Run open-source AI models privately on your own device
Stars
77,347Last commit
1 year agoLicense
MIT
GPT4All runs open-source language models locally on Windows, macOS, and Linux with no cloud dependency, keeping your data on your machine.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
50,139Last commit
10 hours agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
AI-powered research notebook with full privacy control
Stars
Stars growing over the last 30 days41,375Last commit
1 day agoLicense
MIT
Combines AI summarization, podcast generation, and multi-format content ingestion for researchers and students who want to keep their data private.
Open Source Alternative to:
Run AI models locally or connect to cloud, privately
Stars
45,332Last commit
10 hours agoLicense
Apache-2.0
Jan runs open-source AI models on your own hardware or connects to cloud providers like OpenAI, Anthropic, and Google, keeping your conversations off third-party servers.
Open Source Alternative to:
Open source LLM engineering platform for AI-powered applications
Stars
Stars growing over the last 30 days36,898Last commit
10 hours agoLicense
MIT
Langfuse provides tracing, evaluations, prompt management, and analytics to debug and improve LLM applications.
Open Source Alternative to:
Open-source LLM tracing & evaluation for AI optimization
Stars
Stars growing over the last 30 days12,106Last commit
10 hours agoLicense
Elastic-2.0
Open-source platform for LLM tracing, evaluation, and optimization. Features automatic instrumentation, prompt playground, and real-time AI application monitoring.
Open Source Alternative to:
Monitor LLM performance with open-source observability
Stars
7,516Last commit
16 hours agoLicense
Apache-2.0
Open-source observability platform for LLMs using OpenTelemetry. Monitor performance, track costs, and debug AI applications with just 2 lines of code.
Open Source Alternative to:
Build reliable AI apps with comprehensive observability
Stars
6,254Last commit
6 days agoLicense
Apache-2.0
Open-source platform for logging, monitoring, and debugging LLM applications. Route, debug, and analyze AI apps with comprehensive observability tools.
Open Source Alternative to:
Build, automate, and improve AI agents with your team
Stars
4,881Last commit
10 hours agoLicense
MIT
Open-source workspace for building and running AI agents: chat-driven development, scheduling, human-in-the-loop approvals, versioning, and tracing. Self-host or use the cloud.
Open Source Alternative to:
Full observability for AI agents in production
Stars
4,742Last commit
15 hours agoLicense
MIT
Open-source platform for monitoring AI agents: captures traces, surfaces failure patterns, alerts on issues, and helps you verify fixes with automated evals.
Open Source Alternative to:
Simulation-based testing and evaluation for AI agents
Stars
5,004Last commit
10 hours agoLicense
Apache-2.0
Tests AI agents through multi-turn simulations, LLM-based scoring, and production tracing so teams can ship reliable agents with confidence.
Open Source Alternative to:
AI-powered platform for engineering LLM products
Stars
Stars growing over the last 30 days3,471Last commit
11 hours agoLicense
Apache-2.0
Laminar is an open-source platform that helps collect, understand, and utilize data for building high-quality LLM applications.
Open Source Alternative to:
Python, SQL, and AI in a collaborative data notebook
Stars
3,028Last commit
1 day agoLicense
Apache-2.0
Deepnote is an open-source collaborative notebook for data analysts and scientists, combining Python, SQL, AI assistance, and live sharing in one cloud platform.
Open Source Alternative to:
Monitor, debug, and scale LLM applications with ease
Stars
Stars growing over the last 30 days2,899Last commit
10 hours agoLicense
Apache-2.0
Open-source observability platform for GenAI and LLM applications. Real-time monitoring, distributed tracing, prompt management, and AI model evaluation built on OpenTelemetry.
Open Source Alternative to:
GPU orchestration across clouds, Kubernetes, and on-prem
Stars
2,299Last commit
13 hours agoLicense
MPL-2.0
Unified control plane for AI compute that provisions and schedules GPU workloads across any cloud, Kubernetes cluster, or bare-metal server.
Open Source Alternative to:
Serverless GPU compute with sub-second cold starts
Stars
1,842Last commit
10 hours agoLicense
AGPL-3.0
Run GPU inference, task queues, and sandboxes on serverless infrastructure with sub-second cold starts, autoscaling, and support for your own AWS, GCP, or bare metal.
Open Source Alternative to:
Experiment tracking platform for ML teams
Stars
394Last commit
8 months agoLicense
Apache-2.0
Open source ML experiment tracking platform with parameter and gradient logging, media tracking, real-time alerts, and full Weights & Biases API compatibility.
Open Source Alternative to:
LLM observability: cost, latency, and traces in one place
Stars
13Last commit
1 day agoLicense
MIT
Drop-in observability platform for OpenAI, Anthropic, and Gemini that logs every request, tracks costs, traces agent workflows, and flags anomalies and PII.
Open Source Alternative to:
Trace, evaluate, and protect every LLM call in one SDK
Stars
12Last commit
11 hours agoLicense
AGPL-3.0
An AI gateway that sits on the request path to handle PII redaction, prompt injection blocking, tracing, evals, and cost tracking for OpenAI, Anthropic, and Gemini.
Open Source Alternative to:
Postiz – The All-in-One agentic social media scheduling tool
Stellar Hosted
Highseat