Stellar Hosted – Managed Open Source software hosting in the EU: secure, compliant, fast.

Learn more

Open Source Web Crawlers using Python

A curated collection of the best open source tools for automatically crawling and extracting data from websites. using Python.

Firecrawl

Blazing-fast web crawling for modern developers

Stars
110,903
Forks
7,073
Last commit
15 hours ago

Efficient, scalable web crawler built on Rust. Extract data, monitor sites, and automate web tasks with ease and speed.

Open Source Alternative to:

Browserbase

Crawl4AI

Open-source web crawler optimized for AI and LLMs

Stars
64,301
Forks
6,587
Last commit
7 hours ago

Fast, AI-ready web crawler that generates clean markdown for RAG pipelines. Features adaptive crawling, structured extraction, and advanced browser control.

Open Source Alternative to:

Browserbase

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit

Stars
Forks
Last commit