webscraping.ai valid
WebScraping.AI
Web scraping API: send a URL, get back rendered HTML, clean text, CSS-selected fragments, or AI answers and structured JSON extracted from the page. JavaScript rendering, rotating datacenter/residential/stealth proxies and retries are handled server-side. Also parsed Google search results (`/serp`) and structured data for supported social sites (`/data`).
For agents: Web scraping API returning rendered HTML, clean text, CSS-selected fragments, AI-extracted fields and Google SERP results, with proxies and JS rendering handled
web scraping apiheadless browserproxy rotationserp apihtml extractionstructured data
- Authority
- 51 reach 6 · trust 5 · quality 25 · freshness 15
- Tier
- listed
- llms.txt
- https://webscraping.ai/llms.txt
- llms-full.txt
- https://webscraping.ai/llms-full.txt
- Sections
- 8
- Links
- 162
- Size
- 37945 bytes
- Popularity rank
- #631894
- Categories
- AI and machine learning · APIs and SDKs · Developer tools
- For agents
- Interface: a documented API or declared MCP endpoint an agent can call
- Audience
- developers · agents can: read docs, call api · en
- First seen
- 2026-10-19T10:44:24.832Z
- Last checked
- 2026-10-21T10:30:33.640Z
- Badge
Their llms.txt
The beginning of the file, from our validated copy saved 2026-10-21T10:30:29.094Z. Cached copy · Source on webscraping.ai
# WebScraping.AI
> Web scraping API: send a URL, get back rendered HTML, clean text, CSS-selected fragments, or AI answers and structured JSON extracted from the page. JavaScript rendering, rotating datacenter/residential/stealth proxies and retries are handled server-side. Also parsed Google search results (`/serp`) and structured data for supported social sites (`/data`).
- API base URL: `https://api.webscraping.ai`; every endpoint is a GET authenticated with the `api_key` query parameter.
- Pricing is in credits per successful request (failed requests are free). The free plan includes 2,000 credits per month (2 concurrent requests); see [Rules & Pricing](https://webscraping.ai/docs/rules.md) for per-option costs.
- Machine-readable spec: [OpenAPI (YAML)](https://webscraping.ai/openapi.yml), [OpenAPI (JSON)](https://webscraping.ai/openapi.json).
- Remote MCP server for AI assistants: `https://mcp.webscraping.ai/mcp` (Streamable HTTP, OAuth sign-in, no API key needed).
- Official SDKs for Python, JavaScript, PHP, Ruby, Go, Java and C# / .NET, an n8n node, and a CLI with an agent skill: see [SDKs](https://webscraping.ai/docs/sdks.md) and [Using with AI Tools](https://webscraping.ai/docs/ai-tools.md).
- Every page on this site is available as Markdown: append `.md` to its path (`/index.md` for the home page) or request it with `Accept: text/markdown`.
## Docs
- [Full API documentation](https://webscraping.ai/docs.md): every endpoint, parameter and error on one page
- [Full documentation as one file](https://webscraping.ai/llms-full.txt): the API docs plus the main product pages
- [Quick Start](https://webscraping.ai/docs/quick-start.md)
- [Authentication](https://webscraping.ai/docs/authentication.md)
- [Base URL](https://webscraping.ai/docs/base-url.md)
- [Rules & Pricing](https://webscraping.ai/docs/rules.md)
- [Success Rates & Retrying](https://webscraping.ai/docs/success-rates.md)
- [Best Practices](https://webscraping.ai/docs/best-practices.md)
- [Sending Cookies](https://webscraping.ai/docs/cookies.md)
- [Official SDKs](https://webscraping.ai/docs/sdks.md)
- [Using with AI Tools](https://webscraping.ai/docs/ai-tools.md)
- [Proxy Mode](https://webscraping.ai/docs/proxy-mode.md)
- [Ask Questions About a Page](https://webscraping.ai/docs/ai-question.md)
- [Extract Structured Fields](https://webscraping.ai/docs/ai-fields.md)
- [Get Page HTML](https://webscraping.ai/docs/html.md)
- [Get Page Text](https://webscraping.ai/docs/text.md)
- [Get Selected HTML](https://webscraping.ai/docs/selected.md)
- [Get Multiple Selected Areas](https://webscraping.ai/docs/selected-multiple.md)
- [Search Results (SERP)](https://webscraping.ai/docs/serp.md)
- [Structured Data](https://webscraping.ai/docs/data.md)
- [Account Information](https://webscraping.ai/docs/account.md)
- [Error Codes](https://webscraping.ai/docs/errors.md)
- [Parameters Reference](https://webscraping.ai/docs/parameters.md)
- [Full API Reference](https://webscraping.ai/docs/openapi.md)
- [
…Related sites
Other listed sites in AI and machine learning. See the whole category →
| Domain | Title | Category | Country | Authority | llms.txt | llms-full.txt | Save |
|---|---|---|---|---|---|---|---|
| sourcey.com | Sourcey | interface AI and machine learning AI agents and MCP | 70 | ✓ | ✓ | ||
| gofrantic.com | Frantic | interface AI and machine learning AI agents and MCP | 61 | ✓ | |||
| intellweb.ai | Intellweb | documentation Ecommerce Productivity +3 | US | 61 | ✓ | ✓ | |
| agentwebmap.com | Agent Web Map | interface AI and machine learning AI agents and MCP +1 | FR | 60 | ✓ | ✓ | |
| cloudflare.com | Cloudflare | interface AI and machine learning Cybersecurity +1 | 85 | ✓ | ✓ | ||
| adobe.com | Adobe Tools | transactional AI and machine learning Design +1 | FR | 75 | ✓ | ||
| taboola.com | Taboola.com | documentation AI and machine learning Marketing | 75 | ✓ | ✓ | ||
| opera.com | The Opera browser overview | documentation Infrastructure AI and machine learning +2 | NL | 74 | ✓ |
Machine-readable: GET /v1/resolve?q=webscraping.ai · GET /v1/domain/webscraping.ai · this page as Accept: application/json or text/markdown · MCP lookup_domain
Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.
Own or manage this website?
If you own or manage webscraping.ai, you can update its listing and review file health, discovery activity and score history.
Manage this listing — optional
What verification unlocks
See how your site is discovered—and control what agents see.
- Control what agents see
Edit your listing description, tags, public contact details and additional llms.txt paths. Publish your terms for automated access, attribution and pricing.
- Make your profile eligible for search
A valid, listed, unblocked, verified profile can appear in our publisher sitemap and become eligible for search-engine visibility. Search engines decide what to index and rank.
- Appear in our llms-full.txt
Eligible verified publishers are listed by name, with their file links and summaries, in the service overview agents can read.
Free. No account registration, backlink or payment required. Public discovery remains available without verification.
Let your coding agent verify it
If your coding agent is authorized to manage this website, paste these instructions into it.
Claim my listing on Agent Web Map for webscraping.ai:
1. POST https://agentwebmap.com/v1/sites/webscraping.ai/claims and keep claim_id and claim_secret private.
2. Publish the proof: add the line "AgentWebMap ownership: <token>" (token from step 1) to https://webscraping.ai/llms.txt, or use the DNS, meta-tag or well-known method the response describes.
3. POST https://agentwebmap.com/v1/sites/webscraping.ai/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/webscraping.ai).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verifyCorrections or removal requests: contact the directory.