JSON API

Free, no authentication for reads. Please cache responses and keep request rates reasonable.

Listings are ranked by an authority score (0–100) built only from free, verifiable signals: reach 35 (best rank across the Tranco and Majestic Million top lists, +2 when on both), trust 25 (verified ownership 15, domain age from the registry's RDAP record up to 5, live homepage 5), file quality 25 (title, summary, sections, links, size, llms-full.txt; minus dead sampled links or duplicated content), freshness 15 (how recently the file was last seen valid). Tiers: featured ≥ 70, listed ≥ 45, else unlisted (direct lookup only). An llms.txt served identically by three or more domains is treated as a template and kept unlisted.

GET /v1/stats

curl https://agentwebmap.com/v1/stats

GET /v1/domains

Listed domains, ranked. Params: page, per_page (max 100), include_unlisted=true.

curl "https://agentwebmap.com/v1/domains?page=1&per_page=50"

GET /v1/domain/:domain

curl https://agentwebmap.com/v1/domain/anthropic.com

GET /v1/search

Full-text search over domain, title, summary, description and tags. Params: q, page, per_page, include_unlisted.

curl "https://agentwebmap.com/v1/search?q=documentation"

POST /v1/sites

Submit a domain for crawling. The response includes verification instructions so an agent can continue without another call.

curl -X POST https://agentwebmap.com/v1/sites \
  -H "content-type: application/json" \
  -d '{"domain": "example.com"}'

Verification

GET /v1/sites/:domain/verification returns the domain's token and the four accepted methods. Place the token using any one of them, then call POST /v1/sites/:domain/verify. On success the response includes an API key (shown once; calling verify again issues a new one). Verification is re-checked on every crawl and lapses after 14 days of continuous failure.

{
  "token": "awm_61cc0c993653afd1d471ba74a7c441f3",
  "methods": [
    {
      "method": "llms_txt",
      "snippet": "- [Agent Web Map](https://agentwebmap.com/domain/example.com?verify=awm_61cc0c993653afd1d471ba74a7c441f3)"
    },
    {
      "method": "dns",
      "snippet": "_agentwebmap.example.com. TXT \"awm-verify=awm_61cc0c993653afd1d471ba74a7c441f3\""
    },
    {
      "method": "meta",
      "snippet": "<meta name=\"agentwebmap-verify\" content=\"awm_61cc0c993653afd1d471ba74a7c441f3\">"
    },
    {
      "method": "well_known",
      "snippet": "https://example.com/.well-known/agentwebmap-verify.txt  ->  awm_61cc0c993653afd1d471ba74a7c441f3"
    }
  ]
}
curl -X POST https://agentwebmap.com/v1/sites/example.com/verify
# → {"domain":"example.com","verified":true,"via":"llms_txt","api_key":"awm_key_…"}

Owner endpoints (Bearer API key)

PATCH /v1/sites/:domain — set description (≤500 chars), tags (≤10), nested_llms_txt (≤20 same-site paths such as /docs/llms.txt), contact.

curl -X PATCH https://agentwebmap.com/v1/sites/example.com \
  -H "authorization: Bearer awm_key_…" -H "content-type: application/json" \
  -d '{"description": "…", "tags": ["developer-tools","docs"], "nested_llms_txt": ["/docs/llms.txt"]}'

POST /v1/sites/:domain/recrawl — re-validate immediately instead of waiting for the 30-day cycle (once per 10 minutes).

What verified owners get