Web Data Extraction · Public

firecrawl

Web data extraction project for checking crawl boundaries, permissions, structured output, and recovery behavior.

Web crawlingStructured extractionPermission boundariesData cleanupRecovery

Best fitDevelopers who need web content as structured data or AI context and can manage permission, rate, and quality boundaries.

Check whether this project matches your task before installing it.

What it can doCrawl-permission preflights, structured-output checks, rate control, data cleanup, and recovery guidance

Review the portable capability path.

Before continuingVerify in a sandbox

Do not treat a preview pack as a proven local install.

GitHub snapshot118k stars

7.3k forks · 151 contributors

Official first step Read manual preview Source repository

Publication status · 2026-05-25

What is firecrawl?

firecrawl supports web crawling, structured extraction, or turning web content into AI-usable data.
Best fit: Developers who need web content as structured data or AI context and can manage permission, rate, and quality boundaries.
Not for: Not for users who cannot verify target-site permission, isolate cookies/accounts, or handle anti-bot and data-quality issues.
Capability added to an AI workflow: Crawl-permission preflights, structured-output checks, rate control, data cleanup, and recovery guidance
First safe verification step: Verify crawling, structured output, and stop/retry behavior on public test pages first.
Verification state: source, Quick Start, and sandbox install checks are recorded as passed.
Top risk: The main risk is unverified permissions, rate limits, cookie/account state, or data quality.
Evidence base: https://github.com/firecrawl/firecrawl, https://github.com/firecrawl/firecrawl#readme, Human Manual, Pitfall Log

Quick decision

Use this section to decide whether the project is worth a deeper read.

Best forDevelopers who need web content as structured data or AI context and can manage permission, rate, and quality boundaries.

Match the project to your task before installing it.

CapabilityCrawl-permission preflights, structured-output checks, rate control, data cleanup, and recovery guidance

Web data extraction project for checking crawl boundaries, permissions, structured output, and recovery behavior.

Repositoryfirecrawl/firecrawl

118k stars · 7.3k forks

What it can do

Translate the upstream project into concrete capabilities the user can judge before installing.

Introduction to Firecrawl

Related topics: System Architecture, Search Functionality, Web Scraper Engine

Sources: [README.md](https://github.com/firecrawl/firecrawl/blob/main/README.md)

Project File Structure

Related topics: Introduction to Firecrawl, System Architecture

Sources: [apps/api/package.json](https://github.com/firecrawl/firecrawl/blob/main/apps/api/package.json)

System Architecture

Related topics: Introduction to Firecrawl, API v2 Endpoints

Sources: [apps/api/src/controllers/auth.ts:1-45](https://github.com/firecrawl/firecrawl/blob/main/apps/api/src/controllers/auth.ts)

Search Functionality

Related topics: Web Scraper Engine, API v2 Endpoints

Source: https://github.com/firecrawl/firecrawl / Human Manual

Web Scraper Engine

Related topics: Search Functionality, Agent and Deep Research, API v2 Endpoints

Source: https://github.com/firecrawl/firecrawl / Human Manual

Sources: https://github.com/firecrawl/firecrawl, Human Manual, Project Pack evidence, and downstream validation signals.

Community Discussion Evidence

Project-level external discussion stays visible on the detail page, not only inside the manual.

Stars118k stars

Forks7.3k forks

Contributors151 contributors

Licenseunknown

Community Discussion Evidence

12 source-linked items

Review these external discussions before using firecrawl with real data or production workflows. They are review inputs, not standalone proof that the project is production-ready.

01
[Feat] Support custom HTTP headers in Node.js SDK for self-hosted instan
github / github_issue
02
[Feat] Emit batch scrape failures of each page to webhook
github / github_issue
03
RFC: Lightweight External Memory Capsule Pattern for Firecrawl Agent Wor
github / github_issue
04
[Bug] /interact with language="python" flakily fails with TargetClosedEr
github / github_issue
05
v2.9.0
github / github_release
06
v2.8.0
github / github_release
07
v2.7.0
github / github_release
08
v2.6.0
github / github_release
09
v2.5.0 - The World's Best Web Data API
github / github_release
10
v2.4.0
github / github_release
11
v2.3.0
github / github_release
12
v2.2.0
github / github_release

How to start

Only source-backed commands are shown here. Verify them in an isolated environment first.

Try the prompt first

Test the workflow without installing the upstream project.

preview

Read the Human Manual

Understand inputs, outputs, limits, and failure modes.

manual

Take context to your AI host

Use the compiled assets in your preferred AI environment.

context

Run sandbox verification

Confirm install commands and rollback before using a primary environment.

verify

npx -y firecrawl-cli@latest

Official start command · https://github.com/firecrawl/firecrawl#readme · verified: yes

Human Manual

The English page must expose the real manual, not a short placeholder.

8+ sections · Human Manual

firecrawl Manual

Firecrawl provides four primary capabilities that form the foundation of its web interaction platform:

Open the full manual

firecrawl Human Manual
Table of Contents
Introduction to Firecrawl
Related Pages
Core Features Overview
Search
Scrape
Interact

Introduction to Firecrawl

Related topics: System Architecture, Search Functionality, Web Scraper Engine

Sources: [README.md](https://github.com/firecrawl/firecrawl/blob/main/README.md)

Project File Structure

Related topics: Introduction to Firecrawl, System Architecture

Sources: [apps/api/package.json](https://github.com/firecrawl/firecrawl/blob/main/apps/api/package.json)

System Architecture

Related topics: Introduction to Firecrawl, API v2 Endpoints

Sources: [apps/api/src/controllers/auth.ts:1-45](https://github.com/firecrawl/firecrawl/blob/main/apps/api/src/controllers/auth.ts)

Search Functionality

Related topics: Web Scraper Engine, API v2 Endpoints

Source: https://github.com/firecrawl/firecrawl / Human Manual

Web Scraper Engine

Related topics: Search Functionality, Agent and Deep Research, API v2 Endpoints

Source: https://github.com/firecrawl/firecrawl / Human Manual

AI Context Pack and portable assets

After deciding to continue, take the project context into your own AI host.

Complete pack plus user-owned assets

These files are planning and verification assets for Claude Code, Codex, Gemini, Cursor, ChatGPT, and other AI hosts.

Download complete pack Read Human Manual

BundleComplete Project Pack AssetAI Context Pack AssetBoundary & Risk Card AssetHuman Manual AssetPitfall Log AssetPrompt Preview AssetQuick Start ManifestPROJECT_PACK.json

Preflight checks

Treat this page as a planning asset, not proof that your local environment is ready.

The manual is generated from source-linked project files and Doramagic validation signals.
Community evidence warnings stay visible instead of being converted into marketing claims.
This English page is indexable because the locale quality gate passed and explicit English index approval is enabled.
Use the upstream repository as the final authority for installation commands, license, and version-specific behavior.