hypersolutions.co Hyper Solutions
interface APIs and SDKs DE 52 ✓
prosopo.io Prosopo
documentation Cybersecurity NL 55 ✓ ✓
scraperapi.com ScraperAPI
interface AI and machine learning APIs and SDKs +2 US 58 ✓
scrape.do Scrape.do
interface APIs and SDKs DO 51 ✓
scrapeops.io ScrapeOps
interface Data and analytics APIs and SDKs GB 53 ✓ ✓
crawlbase.com Crawlbase
interface Infrastructure AI agents and MCP +1 — 60 ✓ ✓
zenrows.com Zenrows
interface AI and machine learning AI agents and MCP +2 ES 61 ✓ ✓
proxyrack.com Proxyrack
interface Infrastructure APIs and SDKs US 60 ✓ ✓
botbye.com BotBye
transactional Cybersecurity — 54 ✓
zyte.com Zyte – The complete toolbox for web scraping, data extraction, and proxy infrastructure at scale. Trusted by thousands of companies worldwide.
interface APIs and SDKs Developer tools +1 IE 55 ✓
atmosvere.eu Atmosvere
documentation Cybersecurity FR 49 ✓
gologin.com Gologin LLC: The most useful information about data protection and anonymous Internet surfing
documentation — 57 ✓
proxies.sx PROXIES.SX
interface Developer tools Infrastructure +1 SX 57 ✓ ✓
bothunt.ru BotHunt
documentation Cybersecurity RU 56 ✓ ✓
poems.com.sg Block known aggressive/bad bots
documentation SG 46 ✓
captchasonic.com CaptchaSonic
interface APIs and SDKs — 54 ✓ ✓
botpoison.com Botpoison
interface Cybersecurity APIs and SDKs — 56 ✓ ✓
scrapingpros.com Scraping Pros - Fully Managed Web Scraping Services
documentation Data and analytics — 46 ✓
morphaesthetic.com Morph Aesthetics
documentation Healthcare IN 48 ✓
nethris.com Access granted for GPTBot
documentation GB 48 ✓
pandauth.com Panda Key System (PandaAuth)
documentation Cybersecurity Developer tools — 55 ✓ ✓
friendlycaptcha.com Friendly Captcha - Privacy-First Bot Protection & Invisible CAPTCHA Alternative
interface Cybersecurity APIs and SDKs DE 63 ✓
acibademinternational.com Acibadem Hospitals Group
transactional Healthcare TR 62 ✓ ✓
titannet.io Titan Network
transactional Infrastructure Crypto and Web3 HK 50 ✓
vaperanger.com Allow all known major AI and LLM bots to crawl the entire site
informational US 45 ✓
marsproxies.com MarsProxies
transactional Infrastructure — 61 ✓ ✓
cointraffic.com Cointraffic
documentation Marketing Crypto and Web3 EE 56 ✓ ✓
joinmassive.com Massive
interface APIs and SDKs Data and analytics US 64 ✓
anycast.ac anycast.ac
documentation Cybersecurity Infrastructure DE 46 ✓
spamkill.co SpamKill
documentation Cybersecurity — 47 ✓
getbotlens.com BotLens
interface Marketing Data and analytics US 48 ✓
bitdefender.com Bitdefender | Global Leader in Cybersecurity
transactional Cybersecurity — 67 ✓
captchafox.com CaptchaFox
documentation Infrastructure Cybersecurity FR 56 ✓
hanwhadatacenters.com Hanwha Data Centers
documentation Infrastructure Energy and utilities US 46 ✓
icade.fr Specific rules for AI crawlers
informational FR 47 ✓
ipfighter.com IPFighter - Check your IP score | Advanced IP fraud detection
transactional Cybersecurity — 53 ✓
hashe.com Hashe Computer Solutions (Pvt) Ltd.
informational AI and machine learning Developer tools +2 FR 50 ✓
cdnetworks.com 规则组 1: 针对所有爬虫的通用规则
documentation FR 48 ✓
eventualhealthcare.com Eventual Health Care
informational Healthcare SG 46 ✓
cheq.ai CHEQ
documentation Cybersecurity — 60 ✓
mgretailer.com Sensitive paths
— 53 ✓
ipfoxy.com IPFoxy
transactional Ecommerce Infrastructure — 58 ✓
dirjournal.com DirJournal: The Global Business Trust Graph
documentation US 54 ✓ ✓
rjourney.com rjourney.com
informational Travel US 49 ✓
reelo.io Reelo
documentation Marketing US 46 ✓
posbicore.com POSBICORE
informational AI and machine learning Cybersecurity — 46 ✓ ✓
diyanretail.com Diyan Retail
transactional Ecommerce FR 54 ✓
bjkpfw.com Network Security Verification
informational ZA 46 ✓
cymulate.com Cymulate
documentation Cybersecurity — 58 ✓
aksysgames.com Allow AI search assistants
documentation US 47 ✓
freehost.com.ua Разрешаем OpenAI GPTBot (ChatGPT)
documentation UA 50 ✓
castle.io Castle
interface AI and machine learning Cybersecurity +2 US 55 ✓
kitecyber.com Kitecyber
informational AI and machine learning Cybersecurity US 47 ✓
wati.io This means all AI crawlers are disallowed from using your content.
documentation Communications — 45 ✓
lra.gov.lr Liberia Revenue Authority
documentation Government and public services Finance and payments LR 46 ✓
apllogistics.com APL Logistics
documentation Ecommerce Logistics and shipping US 56 ✓
hcaptcha.com hCaptcha
transactional AI and machine learning Cybersecurity — 71 ✓
nohacks.co No Hacks - AI Agent Guidelines
informational AI and machine learning Media +1 — 51 ✓ ✓
duoplus.net DuoPlus Cloud Phone - Anti-Detection Cloud Phone | Mobile Multi-Account Management Expert | TikTok Account Operations | Leader in Antidetect Browser and Cloud Phone
transactional Marketing SG 58 ✓
phishprotection.com Phish Protection
documentation Cybersecurity US 56 ✓ ✓
twinstrata.com Aishwar Babber
informational AI and machine learning — 48 ✓
curator.pro Curator
documentation Cybersecurity RU 53 ✓ ✓
timesofgames.com Times Of Games
informational Gaming Media — 47 ✓
nostra.ai Nostra
transactional Ecommerce Cybersecurity US 52 ✓
botfaqtor.ru llms.txt for botfaqtor.ru
RU 46 ✓
cysinfo.com Cysinfo
documentation Cybersecurity Education +1 FR 46 ✓
buyproxies.org Buy proxies
documentation Infrastructure Developer tools — 48 ✓
wecareindia.com We Care India
documentation Healthcare FR 47 ✓
jmssjgzsx.com Network Security Verification
informational Cybersecurity ZA 46 ✓
jeuliajoias.com Jeulia - Love life,love me - Anéis de noivado & Alianças de casamento
transactional Ecommerce Fashion and beauty — 51 ✓
afixer.app X Shield: IP Country Blocker
documentation Infrastructure Ecommerce +1 — 47 ✓
wuwas.com Wuwas.com
informational Marketing FR 51 ✓
vercel.com Vercel
interface Infrastructure US 67 ✓
minecraftafk.com MinecraftAFK Docs
transactional Gaming — 57 ✓ ✓
guardsquare.com Frequently Asked Questions
documentation Cybersecurity Developer tools US 61 ✓
comarcalcv.com ComarcalCV: ComarcalCV es un medio de información local y comarcal de la Comunitat Valenciana. Noticias, sucesos, reportajes de los municipios de las provincias de Alicante, Castellón y Valencia.
informational Media — 47 ✓
gardensuperstore.com.au The Garden Superstore
transactional Home and garden AU 46 ✓ ✓
cross-browser.org Cross browser - best antidetect browser and privacy tools
informational Cybersecurity — 51 ✓
aurigin.ai Aurigin.ai
interface AI and machine learning Cybersecurity — 50 ✓ ✓
incidentsecure.org Incident Security Authority
documentation Cybersecurity — 47 ✓
ls.app Linken Sphere
documentation Cybersecurity IE 58 ✓
troomi.com Troomi Wireless
documentation Consumer electronics US 56 ✓
ltcillinois.org Learning Technology Center
documentation Education Cybersecurity US 54 ✓
vpnunlimited.com VPN Unlimited (vpnunlimited.com)
transactional Infrastructure Cybersecurity — 58 ✓
checkmarx.com Checkmarx
documentation AI and machine learning Cybersecurity +1 US 61 ✓
coywolf.com Coywolf
documentation AI and machine learning Personal sites and blogs — 52 ✓
bo2k.com BO2K
informational Cybersecurity FR 51 ✓
delta-executor.org Delta Executor
documentation Gaming — 48 ✓
runlayer.com Runlayer
documentation AI and machine learning AI agents and MCP +1 US 61 ✓ ✓
musicianscart.com Musicians Cart
transactional Marketing Ecommerce +1 FR 51 ✓
promptarmor.com https://www.promptarmor.com/llms.txt
documentation AI and machine learning Cybersecurity US 58 ✓
greasemonkey.com Grease Monkey®
informational Automotive US 54 ✓
ffgrv.com Future Forecasters Group
documentation NL 54 ✓ ✓
dropified.com Dropified
informational Ecommerce — 56 ✓
africanews24.com Africa News 24
documentation Media — 55 ✓
proactivemgmt.com Proactive
documentation US 48 ✓
nullcon.net Nullcon
informational Cybersecurity Events and conferences — 53 ✓
infosectoday.com InfosecToday.com
informational Cybersecurity FR 51 ✓
profitsquad.co.uk ProfitSquadUK
informational Media GB 45 ✓
upclass.ai Upclass
documentation Education Finance and payments +1 IN 49 ✓
braincap.ro Braincap (BRAINCAP SRL)
documentation Infrastructure Cybersecurity RO 51 ✓ ✓
californiawaterblog.com California WaterBlog
informational Science and research US 56 ✓
avonsecurityproducts.com Avon Security Products: Avon Security Products — Canada's source for ID card printers, access control cards, and badge printing supplies.
transactional US 47 ✓
doralhw.org Doral Health & Wellness: Doral is committed to providing high quality health care and wellness to all our community’s we serve. With additional focus on disease prevention, we strive for a healthi
documentation Healthcare US 51 ✓
getroamify.com Roamify
transactional Travel US 57 ✓ ✓
trendaisecurity.com Trend AI Security
documentation Cybersecurity AI and machine learning — 62 ✓ ✓
fongo.com Fongo
transactional Telecom Communications — 60 ✓
forerunnershealthcare.com Forerunners Healthcare
documentation Healthcare Travel US 51 ✓
medstarhealth.org MedStar Health Content Index
informational Healthcare US 61 ✓
ens-newswire.com Environment News Service (ENS)
informational Media US 60 ✓
2minutemedicine.com 2 Minute Medicine
transactional AI and machine learning Healthcare US 56 ✓
amlbot.com AMLBot
interface Finance and payments Cybersecurity HK 56 ✓
sanguisughe.com SanguiSughe News
informational Gaming FR 46 ✓
nyasatimes.com Malawi News | Breaking News, Politics, Business & Sports | Nyasa Times
informational Media — 58 ✓ ✓
belkasoft.com Belkasoft: Intelligent Software for Digital Forensics and Cyber Incident Response
documentation Cybersecurity US 56 ✓
hempthy.com Hempthy CBD UK
transactional Industry and manufacturing Ecommerce GB 48 ✓
unik-seo.com UniK SEO
documentation Marketing FR 57 ✓ ✓
larrysalibra.com Larry Salibra
informational Personal sites and blogs — 55 ✓ ✓
paasnational.com PAAS National
transactional Healthcare US 46 ✓
illinitechs.com Illini Tech Services
documentation Cybersecurity Local business — 51 ✓ ✓
nile1.com NILE1
informational Media US 52 ✓
mediaite.com Mediaite
informational Media — 60 ✓
horsesidevetguide.com Horse Side Vet Guide
informational Agriculture US 46 ✓
vpnalert.com vpnAlert
informational Infrastructure Cybersecurity — 57 ✓
jpmanage.net JP & Me Management
informational Real estate US 46 ✓
iankhan.com Ian Khan | AI Keynote Speaker for Hire | Futurist, Author, Thinkers50
documentation AI and machine learning Events and conferences US 55 ✓ ✓
needsomefun.net NSF Tech News
informational Media — 46 ✓
anomali.com anomali.com llms.txt
documentation Cybersecurity AI and machine learning US 59 ✓
legalexaminer.com The Legal Examiner
informational Legal AI and machine learning +1 DE 62 ✓ ✓
raininfotech.com Rain Infotech
documentation AI and machine learning GB 48 ✓
joindeleteme.com DeleteMe
transactional Cybersecurity US 55 ✓
originhq.com Origin Technology
documentation AI and machine learning Cybersecurity +1 US 55 ✓
yfsmagazine.com YFS Magazine
informational Media — 55 ✓ ✓
generateprompt.ai GeneratePrompt.ai
informational AI and machine learning DE 54 ✓ ✓
netdiligence.com NetDiligence
documentation Cybersecurity — 53 ✓
affiliatebay.net AffiliateBay
informational AI and machine learning US 48 ✓
amadpart.com llms.txt
informational Ecommerce IR 48 ✓
bigamart.com BigaMart
transactional — 48 ✓