A curated collection of the best open source projects tagged "Gpu". Each listing includes a website screenshot along with a detailed review of its features.
High-throughput, memory-efficient serving engine for LLMs
Stars
92,916
Last commit
3 hours ago
License
Apache-2.0
Inference and serving engine for large language models, built for speed and hardware efficiency with an OpenAI-compatible API and support for a wide range of open models.
Serverless GPU compute with sub-second cold starts
Stars
1,799
Last commit
11 hours ago
License
AGPL-3.0
Run GPU inference, task queues, and sandboxes on serverless infrastructure with sub-second cold starts, autoscaling, and support for your own AWS, GCP, or bare metal.