Skip to content
ToolOpen-source agent

crawl4ai

Web crawler and hosted API that turns URLs into LLM-ready Markdown or JSON

Pricing
Freemium
Platforms
Web, API, self-hosted
GitHub stars
87.5K
Last commit
4 days ago
Language
Python
License
Apache-2.0

About crawl4ai

crawl4ai is a freemium AI tool for web search and scraping. It is available on Web, API, self-hosted and CLI.

Crawl4AI began as an Apache-2.0 Python crawler and now also has a hosted edition, Crawl4AI Cloud, run by the same team. The hosted service runs the browsers and proxies, so a single API key covers scraping, extraction, search, search-result pages, site scans and change monitoring.

Billing is by credit: a plain page costs 0.1 credit, a page rendered in a real browser 0.5 and a hard site 2, and failed calls are not charged. Extraction can be driven by a plain-language instruction that returns typed JSON, and the site lists 93 ready-made recipes for leads, launches, shops and local search.

The cloud offers US, Singapore and EU regions and says GDPR and PDPA data processing agreements are available, with SOC 2 in progress.

What it does

Markdown and JSON output
Converts any URL into clean Markdown or structured JSON.
Instruction-based extraction
Describe the fields you want in plain words and receive typed JSON.
Ready-made recipes
93 recipes cover leads, launches, social profiles, shops and local search.
Site scan and crawl
Lists a site's URLs from sitemaps and crawls it level by level inside the site.
Change monitoring
Watches a page and alerts when what you asked about changes.
Credit-based billing
Failed calls cost nothing; pages are priced by how hard they were to fetch.

Good for

  • Feeding clean page Markdown into a RAG pipeline
  • Extracting structured product or lead data from websites
  • Monitoring a competitor's pricing page for changes
  • Mapping every URL of a site before a bulk crawl

Pricing

Freemium

1,000 free credits a month; extra credits sold in packs (from $1 per 1k credits)

Prices change. Check crawl4ai.com before you buy.

crawl4ai FAQ

Is there a free tier?

Yes, the hosted service advertises 1,000 free credits a month with no credit card.

What does a page cost?

A plain page is 0.1 credit, a real-browser page 0.5 and a hard site 2; failed calls are free.

Is the engine open source?

The repository is Apache-2.0 licensed and the cloud uses the same engine as the open-source library.

Is crawl4ai free?

crawl4ai has a free plan, with paid plans for more usage or features.

What are the best alternatives to crawl4ai?

The closest alternatives to crawl4ai are ECC, firecrawl, ponytail, graphify and ragflow.

Alternatives to crawl4ai

  1. Skills, hooks, memory and security layer for coding agent harnesses

    • Freemium
    • Open-source agent
    • 305K GitHub stars
    • AI models and LLM tooling
  2. Web search, scraping and page interaction API for AI agents and developers

    • Freemium
    • Agent framework
    • 202K GitHub stars
    • Web search and scraping
  3. Ruleset that makes AI coding agents write less code that still works

    • Open source
    • Open-source agent
    • 192K GitHub stars
    • AI models and LLM tooling
  4. Code knowledge graph that gives coding assistants shared, source-linked context

    • Freemium
    • Open-source agent
    • 149K GitHub stars
    • Knowledge and memory
  5. Open-source RAG engine with a document pipeline, hybrid search and visual agent workflows

    • Freemium
    • Open-source agent
    • 94.3K GitHub stars
    • Knowledge and memory
  6. Command-line channels that let AI agents read X, Reddit, YouTube and Chinese platforms

    • Open source
    • Open-source agent
    • 112K GitHub stars
    • AI models and LLM tooling