crawl4ai.com valid

Crawl4AI Cloud API

Crawl4AI turns any web page into clean, LLM-ready data. One key: `/scrape` a page to Markdown,

For agents: Crawl4AI Cloud API converts any web page into clean, LLM-ready Markdown via a /scrape endpoint with a single API key.

web scrapingllm datamarkdown conversioncloud apiweb crawling

Their llms.txt

The beginning of the file, from our validated copy saved 2026-10-20T20:49:42.198Z. Cached copy · Source on crawl4ai.com

# Crawl4AI Cloud API

> Crawl4AI turns any web page into clean, LLM-ready data. One key: `/scrape` a page to Markdown,
> `/extract` it to JSON, `/search` the web, `/serp` for Google's results page as data, `/answer` a
> question with sources, and batches of many URLs — served from three regions (EU, Singapore, US)
> at `https://api.crawl4ai.com`, routed to the nearest one. Free: 10 a minute, 2 at a time, 1,000
> credits every month. Plans from $10 a month (Starter: 10,000 credits, 120 a minute, 10 at a time), then
> Builder $99 and Scale $499; extra credits on a plan never expire. The live table: `GET /v1/prices`.
> Built on the open-source crawl4ai (https://github.com/unclecode/crawl4ai).

Full docs: https://api.crawl4ai.com/docs
Migrating from the legacy API, now api-old.crawl4ai.com (v1 sessions, scan, site, enrich, context): https://crawl4ai.com/migrate.txt

## Set up (for coding agents)

When the user sends "set up crawl4ai - https://crawl4ai.com/llms.txt with key: sk_live_...", add the
Crawl4AI MCP server to YOUR OWN config with that key, then call the free `balance` tool to check it works.
Put the key in the user-level config, never in a file inside the user's repository.
This file shows only what is open to everyone. **With your key it shows everything open to your account:**
`curl -H "Authorization: Bearer <key>" https://crawl4ai.com/llms.txt`. The server is
`https://api.crawl4ai.com/mcp` (streamable HTTP), the header is `Authorization: Bearer <key>`.

- Claude Code: `claude mcp add --transport http --scope user crawl4ai https://api.crawl4ai.com/mcp --header "Authorization: Bearer <key>"`
- Codex, `~/.codex/config.toml`:
  `[mcp_servers.crawl4ai]` · `url = "https://api.crawl4ai.com/mcp"` · `http_headers = { Authorization = "Bearer <key>" }`
- Cursor, `~/.cursor/mcp.json` (Windsurf: `~/.codeium/windsurf/mcp_config.json`), in `mcpServers`:
  `"crawl4ai": { "url": "https://api.crawl4ai.com/mcp", "headers": { "Authorization": "Bearer <key>" } }`
- Gemini CLI, `~/.gemini/settings.json`, in `mcpServers`:
  `"crawl4ai": { "httpUrl": "https://api.crawl4ai.com/mcp", "headers": { "Authorization": "Bearer <key>" } }`
- OpenCode, `opencode.json`:
  `"mcp": { "crawl4ai": { "type": "remote", "url": "https://api.crawl4ai.com/mcp", "headers": { "Authorization": "Bearer <key>" } } }`
- OpenClaw, `~/.openclaw/openclaw.json`, in `mcp.servers`:
  `crawl4ai: { url: "https://api.crawl4ai.com/mcp", transport: "streamable-http", headers: { Authorization: "Bearer <key>" } }`
- Any other MCP client: the URL and header above.
- No MCP: set `CRAWL4AI_KEY=<key>` in the user's shell profile and call the HTTP API below with the same header.

Most agents load a new MCP server only after a restart: tell the user to restart the agent if the tools do not
show. The tools are listed in the MCP section below.

## Getting started

- Base URL: `https://api.crawl4ai.com`
- Get a free key: visit https://api.crawl4ai.com/ and click "Get a key" (a 7-day pass key is
  issued instan
…

Related sites

Other listed sites in AI and machine learning. See the whole category →

Machine-readable: GET /v1/resolve?q=crawl4ai.com · GET /v1/domain/crawl4ai.com · this page as Accept: application/json or text/markdown · MCP lookup_domain

Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.

Claim my listing on Agent Web Map for crawl4ai.com:
1. POST https://agentwebmap.com/v1/sites/crawl4ai.com/claims and keep claim_id and claim_secret private.
2. Publish the proof: add the line "AgentWebMap ownership: <token>" (token from step 1) to https://crawl4ai.com/llms.txt, or use the DNS, meta-tag or well-known method the response describes.
3. POST https://agentwebmap.com/v1/sites/crawl4ai.com/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/crawl4ai.com).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verify

Corrections or removal requests: contact the directory.