Open Source Projects tagged "Web Scraping"
A curated collection of the best open source projects tagged "Web Scraping". Each listing includes a website screenshot along with a detailed review of its features.
Turn any website into clean, AI-ready data via API
Stars
Stars growing over the last 30 days199,599Last commit
8 hours agoLicense
AGPL-3.0
API for AI agents to search, scrape, crawl, and interact with the live web, returning clean Markdown, structured JSON, or screenshots from any page.
Open Source Alternative to:
Let AI agents control any browser, at scale
Stars
120,287Last commit
10 hours agoLicense
MIT
Python library that lets AI agents browse the web by giving them real browser control, DOM access, and the ability to interact with any website.
Open Source Alternative to:
LLM-ready web crawler built for AI data pipelines
Stars
87,072Last commit
4 days agoLicense
Apache-2.0
Open-source web crawler and scraper that produces clean, structured output optimized for LLMs, RAG pipelines, and AI agents. Supports async crawling, CSS/XPath/LLM extraction, and stealth browser control.
Open Source Alternative to:
Chat with your documents using any LLM, fully private
Stars
68,052Last commit
2 days agoLicense
MIT
A desktop and self-hostable AI app for chatting with documents, running local LLMs, and using AI agents without writing code or creating an account.
Open Source Alternative to:
No-code web scraping, crawling, and extraction platform
Stars
18,167Last commit
8 hours agoLicense
AGPL-3.0
Turn any website into structured data using a visual recorder, natural language prompts, or API. Includes proxy rotation, scheduling, and webhook integrations.
Open Source Alternative to:
Privacy-first knowledge base with any AI model you choose
Stars
16,804Last commit
13 hours agoLicense
Apache-2.0
Self-hostable NotebookLM alternative that lets teams build searchable knowledge bases from diverse sources, with support for any LLM and no data limits.
Open Source Alternative to:
Self-hosted SERP API for Google, Bing, Yandex, and more
Stars
Stars growing over the last 30 days1,560Last commit
6 days agoLicense
MIT
Query Google, Bing, Yandex, Baidu, DuckDuckGo, and Ecosia through one REST API. Self-host free under MIT, or use the managed cloud option.
Open Source Alternative to:
CodeRabbit – The leading AI Code Review platform. Ship better quality code in 50% less time, with 90% fewer bugs.
Openlane