Open Source Scraping Platforms & SDKs using GitHub
A curated collection of the best open source platforms and SDKs for building web scraping solutions. using GitHub.
LLM-ready web crawler built for AI data pipelines
Stars
86,180Last commit
2 days agoLicense
Apache-2.0
Open-source web crawler and scraper that produces clean, structured output optimized for LLMs, RAG pipelines, and AI agents. Supports async crawling, CSS/XPath/LLM extraction, and stealth browser control.
Open Source Alternative to:
Fast, AI-native web browser built for automation and scraping
Stars
Stars growing over the last 30 days36,942Last commit
14 hours agoLicense
AGPL-3.0
Purpose-built headless browser that delivers 10x faster performance and 10x lower memory usage compared to Chrome headless for web automation and AI workflows.
Open Source Alternative to:
Self-hosted SERP API for Google, Bing, Yandex, and more
Stars
Stars growing over the last 30 days1,499Last commit
5 days agoLicense
MIT
Query Google, Bing, Yandex, Baidu, DuckDuckGo, and Ecosia through one REST API. Self-host free under MIT, or use the managed cloud option.
Open Source Alternative to:
InfluxDB – The purpose-built time series platform for high-velocity ingestion and real-time queries at scale.
Windshift