Open Source Machine Learning Infrastructure Tools
A curated collection of the best open source infrastructure components specifically designed for AI/ML workloads.
Run open-source LLMs locally on your own machine
Stars
180,064Last commit
4 hours agoLicense
MIT
Runs open-source language models locally with a simple setup, plus optional cloud access for larger models when local hardware isn't enough.
Open Source Alternative to:
Run LLMs locally with minimal setup, maximum hardware support
Stars
126,932Last commit
22 minutes agoLicense
MIT
C/C++ inference engine for large language models, supporting quantization, multi-GPU, Apple Silicon, and an OpenAI-compatible server across a wide range of hardware.
Open Source Alternative to:
Run open-source AI models privately on your own device
Stars
77,383Last commit
1 year agoLicense
MIT
GPT4All runs open-source language models locally on Windows, macOS, and Linux with no cloud dependency, keeping your data on your machine.
Open Source Alternative to:
Self-hosted AI runtime for text, voice, vision, and agents
Stars
48,846Last commit
37 minutes agoLicense
MIT
Run LLMs, speech, image generation, and autonomous agents on your own hardware with an OpenAI-compatible API and 60+ swappable backends.
Open Source Alternative to:
AI-powered research notebook with full privacy control
Stars
38,148Last commit
22 hours agoLicense
MIT
Combines AI summarization, podcast generation, and multi-format content ingestion for researchers and students who want to keep their data private.
Open Source Alternative to:
Run AI models locally or connect to cloud, privately
Stars
44,316Last commit
2 hours agoLicense
Apache-2.0
Jan runs open-source AI models on your own hardware or connects to cloud providers like OpenAI, Anthropic, and Google, keeping your conversations off third-party servers.
Open Source Alternative to:
Open source LLM engineering platform for AI-powered applications
Stars
34,161Last commit
45 minutes agoLicense
MIT
Langfuse provides tracing, evaluations, prompt management, and analytics to debug and improve LLM applications.
Open Source Alternative to:
Open-source LLM tracing & evaluation for AI optimization
Stars
11,311Last commit
17 minutes agoLicense
Elastic-2.0
Open-source platform for LLM tracing, evaluation, and optimization. Features automatic instrumentation, prompt playground, and real-time AI application monitoring.
Open Source Alternative to:
Monitor LLM performance with open-source observability
Stars
7,413Last commit
24 days agoLicense
Apache-2.0
Open-source observability platform for LLMs using OpenTelemetry. Monitor performance, track costs, and debug AI applications with just 2 lines of code.
Open Source Alternative to:
Build reliable AI apps with comprehensive observability
Stars
6,130Last commit
4 days agoLicense
Apache-2.0
Open-source platform for logging, monitoring, and debugging LLM applications. Route, debug, and analyze AI apps with comprehensive observability tools.
Open Source Alternative to:
Build, automate, and improve AI agents with your team
Stars
4,679Last commit
25 minutes agoLicense
MIT
Open-source workspace for building and running AI agents: chat-driven development, scheduling, human-in-the-loop approvals, versioning, and tracing. Self-host or use the cloud.
Open Source Alternative to:
Full observability for AI agents in production
Stars
4,619Last commit
2 hours agoLicense
MIT
Open-source platform for monitoring AI agents: captures traces, surfaces failure patterns, alerts on issues, and helps you verify fixes with automated evals.
Open Source Alternative to:
Simulation-based testing and evaluation for AI agents
Stars
3,522Last commit
38 minutes agoLicense
Apache-2.0
Tests AI agents through multi-turn simulations, LLM-based scoring, and production tracing so teams can ship reliable agents with confidence.
Open Source Alternative to:
Python, SQL, and AI in a collaborative data notebook
Stars
3,000Last commit
4 hours agoLicense
Apache-2.0
Deepnote is an open-source collaborative notebook for data analysts and scientists, combining Python, SQL, AI assistance, and live sharing in one cloud platform.
Open Source Alternative to:
AI-powered platform for engineering LLM products
Stars
3,220Last commit
2 hours agoLicense
Apache-2.0
Laminar is an open-source platform that helps collect, understand, and utilize data for building high-quality LLM applications.
Open Source Alternative to:
Monitor, debug, and scale LLM applications with ease
Stars
2,738Last commit
5 hours agoLicense
Apache-2.0
Open-source observability platform for GenAI and LLM applications. Real-time monitoring, distributed tracing, prompt management, and AI model evaluation built on OpenTelemetry.
Open Source Alternative to:
GPU orchestration across clouds, Kubernetes, and on-prem
Stars
2,236Last commit
5 hours agoLicense
MPL-2.0
Unified control plane for AI compute that provisions and schedules GPU workloads across any cloud, Kubernetes cluster, or bare-metal server.
Open Source Alternative to:
Serverless GPU compute with sub-second cold starts
Stars
1,766Last commit
33 minutes agoLicense
AGPL-3.0
Run GPU inference, task queues, and sandboxes on serverless infrastructure with sub-second cold starts, autoscaling, and support for your own AWS, GCP, or bare metal.
Open Source Alternative to:
Experiment tracking platform for ML teams
Stars
392Last commit
6 months agoLicense
Apache-2.0
Open source ML experiment tracking platform with parameter and gradient logging, media tracking, real-time alerts, and full Weights & Biases API compatibility.
Open Source Alternative to:
LLM observability: cost, latency, and traces in one place
Stars
13Last commit
4 days agoLicense
MIT
Drop-in observability platform for OpenAI, Anthropic, and Gemini that logs every request, tracks costs, traces agent workflows, and flags anomalies and PII.
Open Source Alternative to:
Trace, evaluate, and protect every LLM call in one SDK
Stars
7Last commit
9 days agoLicense
AGPL-3.0
An AI gateway that sits on the request path to handle PII redaction, prompt injection blocking, tracing, evals, and cost tracking for OpenAI, Anthropic, and Gemini.
Open Source Alternative to:
Sent – A unified API for sending messages across SMS, RCS, and apps like WhatsApp.
Sent
Sevalla