Browse
Explore shared publisher sections →
45 results for “crawlee”
Country is inferred from the domain or hosting location; it may differ from where a publisher operates. About country data
Actions are inferred from publisher content. Check the publisher’s pages before using a service. About action labels
| Domain | Title | Category | Country | Authority | llms.txt | llms-full.txt | Save |
|---|---|---|---|---|---|---|---|
| proxidize.com | Proxidize | interface Infrastructure APIs and SDKs | 57 | ✓ | ✓ | ||
| crawlee.dev | Crawlee for Python · Fast, reliable Python web crawlers. | documentation Open source projects Developer tools | 50 | ✓ | ✓ | ||
| leboat.com | llm.txt for Le Boat LLM Crawling Guide | US | 50 | ✓ | |||
| legionisci.com | Legionisci.com — AI crawler information | PL | 48 | ✓ | |||
| fintreeindia.com | Fintree - AI Crawler Guidance | IN | 46 | ✓ | |||
| cchcpelink.com | llms.txt - Guidelines for Large Language Models and AI Crawlers | informational AI and machine learning | US | 48 | ✓ | ||
| suvudu.com | llms.txt — AI crawler instructions for this site (November 2025) | US | 45 | ✓ | |||
| weee.com | llms.txt for sayweee.com | US | 46 | ✓ | |||
| cursor.com | Cursor Documentation | US | 58 | ✓ | |||
| cursor.sh | Cursor Documentation | US | 54 | ✓ | |||
| sen7ry.com | Sentry | 46 | ✓ | ✓ | |||
| errortrace.dev | Sentry | US | 57 | ✓ | ✓ | ||
| kyourc.com | KYourC | FR | 48 | ✓ | |||
| gaiaonline.com | Gaia Online - AI Crawler Guidance | US | 50 | ✓ | |||
| crawl4ai.com | Crawl4AI Cloud API | interface AI and machine learning APIs and SDKs | DE | 45 | ✓ | ||
| swatch.com | llms.txt — swatch.com | US | 54 | ✓ | |||
| visitcalgary.com | llms.txt for visitcalgary.com | 45 | ✓ | ||||
| cricketwireless.com | llms.txt for Cricket Wireless | FR | 55 | ✓ | |||
| homeflow.co.uk | Homeflow | GB | 46 | ✓ | |||
| cancer.ca | llms.txt — AI / LLM crawling preferences | CA | 49 | ✓ | |||
| bitkom.org | llms.txt — Machine-readable site overview for AI and research crawlers | DE | 55 | ✓ | |||
| firecrawl.dev | Firecrawl | interface AI and machine learning AI agents and MCP +2 | US | 65 | ✓ | ✓ | |
| conceptfeedback.com | wingify.com | 46 | ✓ | ||||
| visualwebsiteoptimizer.com | wingify.com | US | 66 | ✓ | |||
| openeye.net | llms.txt — OpenEye | CA | 45 | ✓ | |||
| crawlbase.com | Crawlbase | interface APIs and SDKs Data and analytics | 60 | ✓ | ✓ | ||
| privacymanager.io | LiveRamp | 55 | ✓ | ||||
| otis.edu | llms.txt — AI-friendly site summary for LLM crawlers | US | 45 | ✓ | |||
| clinicorp.com | llms.txt for www.clinicorp.com | 45 | ✓ | ||||
| alelo.com.br | ========================= | BR | 46 | ✓ | |||
| ninja-backup.com | NinjaOne | US | 51 | ✓ | |||
| aao.org | ============================================================ | 52 | ✓ | ||||
| ninjarmm.com | NinjaOne | US | 60 | ✓ | |||
| classmanager.com | Class Manager | 48 | ✓ | ||||
| spider.cloud | Spider | interface APIs and SDKs AI agents and MCP | US | 51 | ✓ | ✓ | |
| cricut.com | Cricut | 45 | ✓ | ||||
| monkey.app | ========================================= | 48 | ✓ | ||||
| impact.com | llms.txt for impact.com | 53 | ✓ | ||||
| safe.com | llm.txt for safe.com | US | 47 | ✓ | |||
| getbring.com | llms.txt for getbring.com | US | 46 | ✓ | |||
| mobileiron.com | MobileIron | 62 | ✓ | ✓ | |||
| kosokubus.com | ========================================================= | JP | 45 | ✓ | |||
| clientam.com | ========================================================= | FR | 45 | ✓ | |||
| adbutler.com | ========================================================= | US | 46 | ✓ | ✓ | ||
| agleader.com | llm.txt for agleader.com | documentation AI and machine learning | US | 46 | ✓ |