Open Source Projects tagged "Web Scraping"
A curated collection of the best open source projects tagged "Web Scraping". Each listing includes a website screenshot along with a detailed review of its features.
Turn any website into clean, AI-ready data via API
Stars
192,885Last commit
6 hours agoLicense
AGPL-3.0
API for AI agents to search, scrape, crawl, and interact with the live web, returning clean Markdown, structured JSON, or screenshots from any page.
Open Source Alternative to:
Let AI agents control any browser, at scale
Stars
118,467Last commit
2 days agoLicense
MIT
Python library that lets AI agents browse the web by giving them real browser control, DOM access, and the ability to interact with any website.
Open Source Alternative to:
LLM-ready web crawler built for AI data pipelines
Stars
85,808Last commit
9 hours agoLicense
Apache-2.0
Open-source web crawler and scraper that produces clean, structured output optimized for LLMs, RAG pipelines, and AI agents. Supports async crawling, CSS/XPath/LLM extraction, and stealth browser control.
Open Source Alternative to:
Chat with your documents using any LLM, fully private
Stars
67,299Last commit
2 days agoLicense
MIT
A desktop and self-hostable AI app for chatting with documents, running local LLMs, and using AI agents without writing code or creating an account.
Open Source Alternative to:
No-code web scraping, crawling, and extraction platform
Stars
17,901Last commit
1 day agoLicense
AGPL-3.0
Turn any website into structured data using a visual recorder, natural language prompts, or API. Includes proxy rotation, scheduling, and webhook integrations.
Open Source Alternative to:
Privacy-first knowledge base with any AI model you choose
Stars
16,524Last commit
2 days agoLicense
Apache-2.0
Self-hostable NotebookLM alternative that lets teams build searchable knowledge bases from diverse sources, with support for any LLM and no data limits.
Open Source Alternative to:
Self-hosted SERP API for Google, Bing, Yandex, and more
Stars
1,484Last commit
13 hours agoLicense
MIT
Query Google, Bing, Yandex, Baidu, DuckDuckGo, and Ecosia through one REST API. Self-host free under MIT, or use the managed cloud option.
Open Source Alternative to:
Windshift – The self-hosted platform that replaces Jira.
c15t