lobstr.io valid
lobstr.io
lobstr.io runs ready-made web scrapers in the cloud: 79 scrapers, built and maintained by the lobstr.io team. Give a scraper links, keywords or cities and get structured results as a spreadsheet, delivered to your tools, through the REST API or from an AI agent over MCP.
- Authority
- 46 reach 5 · trust 5 · quality 21 · freshness 15
- Tier
- listed
- llms.txt
- https://www.lobstr.io/llms.txt
- Sections
- 5
- Links
- 79
- Size
- 17303 bytes
- Popularity rank
- #836658
- Country
- United States Hosting inference
- Hosting country
- United States Hosting inference
- Country evidence
- No published address evidence collected yet
- Categories
- AI and machine learning
- For agents
- Documentation: developer or product docs worth reading, nothing to call
- First seen
- 2026-10-25T18:03:20.642Z
- Last checked
- 2026-10-26T04:02:41.270Z
- Badge
Their llms.txt
The beginning of the file, from our validated copy saved 2026-10-26T04:02:40.287Z. Cached copy · Source on lobstr.io
# lobstr.io
> lobstr.io runs ready-made web scrapers in the cloud: 79 scrapers, built and maintained by the lobstr.io team. Give a scraper links, keywords or cities and get structured results as a spreadsheet, delivered to your tools, through the REST API or from an AI agent over MCP.
## Use it from code
- API base URL: https://api.lobstr.io/v1
- Authentication: send the header `Authorization: Token YOUR_API_KEY`. The key is in the dashboard, under API: https://app.lobstr.io/dashboard/api
- Documentation: https://docs.lobstr.io (index for agents: https://docs.lobstr.io/llms.txt)
- The flow: pick a crawler (GET /crawlers), create a squid for it (POST /squids), add its tasks (POST /tasks, or upload a CSV), launch a run (POST /runs), read the results (GET /results).
- Rate limits: 120 requests a minute on /squids and /runs, 90 on /tasks, 2 a second on /results; 429 with Retry-After above that.
## Use it from an AI agent
- MCP server: https://mcp.lobstr.io/mcp (sign in with lobstr.io in the browser, no API key to copy). Works with Claude, ChatGPT, Cursor and any MCP client.
- Setup and the list of tools: https://docs.lobstr.io/docs/mcp
- Running a scraper spends credits: the server shows the cost estimate and asks before it runs.
## Pricing
Credits pay for results; each scraper shows its price per result, and extras (emails, phone numbers) add to it. Only unique results are charged.
- Free: 0 USD a month, 100 credits, 1 slot, results kept 7 days
- Starter: 20 USD a month, 10,000 credits, 2 slots, results kept 28 days
- Pro: 100 USD a month, 100,000 credits, 10 slots, results kept 28 days
- Team: 500 USD a month, 1,000,000 credits, 50 slots, results kept 28 days
Every scraper's price per result, the plans compared and the billing FAQ, in Markdown: https://www.lobstr.io/pricing.md
## Scrapers
- [Google Maps Leads Scraper](https://www.lobstr.io/store/google-maps-leads-scraper): Scrape all local businesses data from a Google Maps Search URL and export into a Google Sheet. Emails included.
- [Google Maps Reviews Scraper](https://www.lobstr.io/store/google-maps-reviews-scraper): Scrape all reviews from a Google Maps Local Business. Extremely fast. For free.
- [Twitter User Tweets Scraper](https://www.lobstr.io/store/twitter-user-tweets-scraper): Extract all tweets from any X/Twitter user — 39 attributes per tweet, no code needed.
- [Sales Navigator Leads Scraper](https://www.lobstr.io/store/sales-navigator-leads-scraper): Extract emails and phones from LinkedIn Sales Navigator at scale — no code, no hassle, easy.
- [YouTube Channel Scraper](https://www.lobstr.io/store/youtube-channel-scraper): Scrape any YouTube channel and get 67 data points including videos, shorts, playlists, comments and contact email.
- [Google Search Scraper](https://www.lobstr.io/store/google-search-scraper): Scrape all Google Search Results from Any Location & Language — organic, ads & People Also Ask included
- [Vinted Products Scraper](https://www.lobstr.io/store/vinte
…Related sites
Category suggestions: other listed sites in AI and machine learning. These share a category. See the whole category →
| Domain | Title | Category | Country | Authority | llms.txt | llms-full.txt | Save |
|---|---|---|---|---|---|---|---|
| gofrantic.com | Frantic | interface AI and machine learning AI agents and MCP | — | 61 | ✓ | ||
| agentwebmap.com | Agent Web Map | interface AI and machine learning AI agents and MCP +1 | FR | 60 | ✓ | ✓ | |
| aiteamguide.com | AITeamGuide | documentation AI and machine learning Marketing +4 | — | 60 | ✓ | ✓ | |
| cloudflare.com | Cloudflare | interface AI and machine learning Infrastructure +1 | — | 85 | ✓ | ✓ | |
| adobe.com | Adobe Tools | transactional AI and machine learning Productivity +1 | FR | 75 | ✓ | ||
| intel.com | Intel Corporation | documentation Industry and manufacturing Consumer electronics +1 | US | 74 | ✓ | ✓ | |
| opera.com | The Opera browser overview | documentation Infrastructure AI and machine learning +2 | NL | 74 | ✓ | ||
| taboola.com | Taboola.com | documentation AI and machine learning Marketing | — | 74 | ✓ | ✓ |
Machine-readable: GET /v1/resolve?q=lobstr.io · GET /v1/domain/lobstr.io · this page as Accept: application/json or text/markdown · MCP lookup_domain
Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.
Own or manage this website?
If you own or manage lobstr.io, you can update its listing and review file health, discovery activity and score history.
Manage this listing — optional
What verification unlocks
See how your site is discovered—and control what agents see.
- See your website in the map
Review your indexed topics, nearby publishers sharing headings, measured listing exposure and map snapshots in a private report for your website.
- Control what agents see
Edit your listing description, tags, public contact details and additional llms.txt paths. Publish your terms for automated access, attribution and pricing.
- Make your profile eligible for search
A valid, listed, unblocked, verified profile can appear in our publisher sitemap and become eligible for search-engine visibility. Search engines decide what to index and rank.
- Appear in our llms-full.txt
Eligible verified publishers are listed by name, with their file links and summaries, in the service overview agents can read.
Free. No account registration, backlink or payment required. Public discovery remains available without verification.
Let your coding agent verify it
If your coding agent is authorized to manage this website, paste these instructions into it.
Claim my listing on Agent Web Map for lobstr.io:
1. POST https://agentwebmap.com/v1/sites/lobstr.io/claims and keep claim_id and claim_secret private.
2. Publish the verification line "AgentWebMap verification: <token>" (token from step 1) in https://lobstr.io/llms.txt, or use the DNS, meta-tag or well-known method the response describes. This only verifies the listing; your website and domain remain yours.
3. POST https://agentwebmap.com/v1/sites/lobstr.io/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/lobstr.io).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verifyCorrections or removal requests: contact the directory.