Stellar Hosted – Managed Open Source software hosting in the EU: secure, compliant, fast.

Learn more

Open Source Web Crawlers using MCP

A curated collection of the best open source tools for automatically crawling and extracting data from websites. using MCP.

Crawl4AI

LLM-ready web crawler built for AI data pipelines

Stars
71,093
Forks
7,307
Last commit
3 days ago

Open-source web crawler and scraper that produces clean, structured output optimized for LLMs, RAG pipelines, and AI agents. Supports async crawling, CSS/XPath/LLM extraction, and stealth browser control.

Open Source Alternative to:

Browserbase

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit