travelonbags.com Travelon
transactional Ecommerce Travel — 55 ✓
spamtitan.com SpamTitan - Email Security and Anti-Spam Protection
transactional Cybersecurity US 57 ✓
antier.com Antier — AI and Blockchain Infrastructure for Enterprises
interface AI and machine learning Infrastructure +1 — 61 ✓ ✓
ivpn.net IVPN
transactional Infrastructure Cybersecurity US 60 ✓
joost.blog joost.blog
documentation Marketing Personal sites and blogs — 57 ✓
bestantiviruspro.org Best Antivirus Pro | Cybersecurity News & Protection
informational Cybersecurity US 49 ✓
poems.com.sg Block known aggressive/bad bots
SG 46 ✓
sankoz.com SANKOZ Custom Workwear
interface Fashion and beauty FR 47 ✓
piketa.com Piketa - AI Access Policy
FI 49 ✓
kosokubus.com =========================================================
JP 45 ✓
whitehousecrawford.com Paus188
informational — 45 ✓
centrocolorpinturas.com Centrocolor Pinturas
transactional Marketing Home and garden BR 46 ✓
clientam.com =========================================================
FR 45 ✓
arandu.com.ar ARANDU: Talabartería Argentina
transactional AR 47 ✓
techlapse.com Techlapse.com – Your Guide to Cybersecurity and Online Safety
informational Cybersecurity — 46 ✓
aloaha.com CodeB Sovereign Communications
interface Authentication and identity ES 51 ✓ ✓
adbutler.com =========================================================
US 46 ✓ ✓
browseract.com BrowserAct - Describe what you need. Get a reusable scraper for any website.
interface AI agents and MCP APIs and SDKs +1 — 54 ✓
blockotoole.com Block O'Toole & Murphy
informational Legal US 47 ✓
holla.world =========================================
— 47 ✓
kaspersky.com.au Kaspersky | Global Cybersecurity & Digital Privacy Solutions
transactional Cybersecurity AU 47 ✓
antiadblockcore.com AntiAdBlock Core
interface Marketing — 46 ✓
monkey.app =========================================
— 48 ✓
owlproxy.com language: English
interface Community Infrastructure +1 HK 56 ✓ ✓
eracks.com eRacks Systems
transactional Infrastructure Consumer electronics CA 46 ✓
html-color-codes.info ========================================
US 45 ✓
alelo.com.br =========================
BR 46 ✓
hawkersco.com Hawkers
transactional Ecommerce Fashion and beauty US 59 ✓
jogsshow.com JOGS Gem & Jewelry Show
informational Events and conferences — 47 ✓
komerza.com Komerza
interface Ecommerce Marketing +1 — 51 ✓
brandsecurity.gmo GMOブランドセキュリティ株式会社
documentation Cybersecurity — 55 ✓ ✓
aoolia.fr Aoolia Optical | Lunettes de vue et lunettes de soleil en ligne
transactional Ecommerce Fashion and beauty FR 47 ✓
candelamedical.com ============================================================
US 49 ✓
cleantalk.org CleanTalk
interface Infrastructure Cybersecurity DE 61 ✓
aao.org ============================================================
— 52 ✓
proxyempire.io ProxyEmpire
documentation Infrastructure — 48 ✓
rbergersen.no R Bergersen
transactional Ecommerce Home and garden NO 47 ✓
sbicard.com =====================================================
IN 49 ✓
loveonpurpose.com Love on Purpose
documentation US 46 ✓
wpmailsmtp.com WP Mail SMTP
documentation Marketing Developer tools +1 — 54 ✓
eltiempo.es =====================================================================
ES 48 ✓
theweathernetwork.com =====================================================================
CA 51 ✓
eliveragroup.com ELIVERA Buy online, Reviews Est. 1988 London, UK
transactional Ecommerce CA 61 ✓ ✓
cybertechnologyinsights.com cybertechnologyinsights.com
documentation Marketing Media +1 US 46 ✓
vpnunlimited.com VPN Unlimited (vpnunlimited.com)
transactional Infrastructure Cybersecurity — 58 ✓
floppydata.com Floppydata
transactional Infrastructure — 57 ✓
crawlbase.com Crawlbase
interface APIs and SDKs Data and analytics — 60 ✓ ✓
cldy.com CLDY
transactional Infrastructure AI and machine learning — 58 ✓
gwtoolbox.com GWToolbox++
documentation Gaming Open source projects — 50 ✓ ✓
kaspersky.co.uk Kaspersky | Global Cybersecurity & Digital Privacy Solutions
transactional Cybersecurity GB 59 ✓
icade.fr Specific rules for AI crawlers
FR 47 ✓
authorityhealthmag.com Authority Health
documentation Healthcare FR 46 ✓
jscrambler.com Jscrambler
documentation Cybersecurity Developer tools US 62 ✓
localdigital.com.au Local Digital
documentation Marketing AU 47 ✓
wpsimplepay.com WP Simple Pay
transactional Finance and payments — 60 ✓ ✓
geonix.com Geonix
transactional Marketing Ecommerce +1 — 57 ✓
ateo.digital Ateo Digital
documentation Infrastructure — 46 ✓
instaproofs.com instaproofs
US 52 ✓
tranzila.com Tranzila
interface Finance and payments IL 62 ✓ ✓
maheshwariandco.com Maheshwari and Co.
informational Legal — 46 ✓
ls.app Linken Sphere
documentation Cybersecurity IE 58 ✓
iplocationblock.com IP Location Block
interface Cybersecurity — 56 ✓
redflag.org.au Red Flag
informational Media AU 56 ✓ ✓
joybang.site Joybang.site
documentation Cybersecurity FI 51 ✓
observatorioblockchain.com Observatorio Blockchain
informational Media Crypto and Web3 — 45 ✓
tuna.am Tuna Документация
documentation Developer tools Infrastructure RU 56 ✓ ✓
timly.com LLMs.txt - AI access policy
— 45 ✓
standsapp.org Stands
documentation US 54 ✓
vegastemcell.com Vega Medical Services
informational Healthcare SG 53 ✓
earthsguards.com حماة الأرض
interface Media LT 48 ✓
notte.cc Notte
interface AI agents and MCP Developer tools US 50 ✓ ✓
bo2k.com BO2K
informational Cybersecurity FR 46 ✓
mgretailer.com Sensitive paths
— 48 ✓
clinicaltrialvanguard.com The Clinical Trial Vanguard
informational Healthcare Media — 54 ✓ ✓
worktime.com ---------------------------------------------------------------------
CA 48 ✓
cosrx.com COSRX Official
transactional Fashion and beauty CA 62 ✓ ✓
pymnts.com PYMNTS
informational Finance and payments Media US 66 ✓
2bfashionable.com 2B Fashionable
transactional Fashion and beauty Ecommerce SG 48 ✓
croxy.com Croxy - Premium Residential Proxy & IP Solutions
transactional Infrastructure — 45 ✓
apify.com Apify
interface Developer tools AI agents and MCP +1 CZ 57 ✓
byteiota.com byteiota
informational AI and machine learning Developer tools US 58 ✓
allout.game All Out
documentation Gaming — 57 ✓
zenrows.com Zenrows
interface AI and machine learning APIs and SDKs +1 — 61 ✓ ✓
bezmezhau.com Bezmezhau DNS
documentation Infrastructure Cybersecurity NL 50 ✓ ✓
kaspersky.co.za Kaspersky | Global Cybersecurity & Digital Privacy Solutions
transactional Cybersecurity ZA 50 ✓
xsolla.com Xsolla
interface Gaming Finance and payments +1 US 63 ✓
lasso.security Lasso Security — llms.txt
US 48 ✓
mergify.com Mergify
interface DevOps and observability Developer tools — 58 ✓
acemarks.com Ace Marks
transactional Ecommerce Fashion and beauty CA 51 ✓ ✓
rapidssl.com RapidSSL
US 56 ✓
dosepharmacy.com Dose Pharmacy
transactional Healthcare US 47 ✓
shiiyu.moe SkyCrypt
— 50 ✓ ✓
capellisport.com Capelli Sport
transactional Sports and fitness Ecommerce CA 60 ✓ ✓
sophos.com Sophos: Defeat Cyberattacks with AI-Native Security
transactional Cybersecurity AI and machine learning FR 69 ✓
pie.org Pie.org: Tools to put users back in control of the internet, starting with Pie Adblock
documentation US 60 ✓
flashid.net FlashID | Antidetect Browser for Secure Multi-Account Management
transactional Ecommerce Cybersecurity +1 SG 47 ✓
geekvillage.com geekvillage.com
informational Community GB 47 ✓
syndmart.com Syndmart
transactional Ecommerce CA 51 ✓ ✓
dxbcoast.com Dubai Coast Trading L.L.C
documentation Industry and manufacturing Local business +1 FR 47 ✓
lionic.com Lionic
documentation Cybersecurity TW 60 ✓
duffelblog.com Duffel Blog
informational Media — 61 ✓ ✓
nohacks.co No Hacks - AI Agent Guidelines
informational AI and machine learning Media +1 — 51 ✓ ✓
cdnetworks.com 规则组 1: 针对所有爬虫的通用规则
documentation FR 48 ✓
bigdug.co.uk LLMS permissions for bigdug.co.uk
GB 46 ✓
theworldwasherefirst.com The World Was Here First
transactional Travel Ecommerce GB 48 ✓
hurra.com encoding: UTF-8
DE 49 ✓
baselgovernance.org Basel Institute on Governance
documentation Education Nonprofit and advocacy DE 59 ✓
permit.io Permit.io
documentation AI and machine learning Authentication and identity +1 US 51 ✓
vamosmisevillafc.com Bloqueo de entrenamiento (Training)
informational DE 46 ✓
justsecurity.org Just Security
documentation Education Cybersecurity +2 — 59 ✓
thaiall.com thaiall.com
documentation US 56 ✓
24metrics.com 24metrics - Click & Affiliate Fraud Prevention
documentation Cybersecurity — 52 ✓
hpharmacy.pk H Pharmacy
informational Healthcare PK 47 ✓
boostclicks.tech BoostClicks
documentation Marketing RU 46 ✓
focusvisioncare.com Focus Vision Care
transactional Healthcare Local business US 46 ✓
fraudlogix.com Fraudlogix
documentation Cybersecurity — 51 ✓
tellor.io tellor
documentation Crypto and Web3 US 49 ✓
vpnalert.com vpnAlert
informational Infrastructure Cybersecurity — 57 ✓
autodidacts.io The Autodidacts
informational Personal sites and blogs US 52 ✓ ✓
cyberghostvpn.com CyberGhost VPN - llms.txt
documentation Infrastructure — 54 ✓
webparsab.com Web Par Sab
informational FR 46 ✓
eabel.com E-Abel
documentation Marketing Industry and manufacturing US 50 ✓
legacycommunityhealth.org Legacy Community Health
informational Marketing Healthcare BR 51 ✓
surfshark.com Surfshark
transactional Cybersecurity NL 66 ✓
surfshark.net Surfshark
transactional Cybersecurity — 55 ✓
sir90hl.com Surfshark
transactional Cybersecurity — 54 ✓
uymgg1.com Surfshark
transactional Cybersecurity — 54 ✓
chairdoor92.com Surfshark
transactional Cybersecurity — 46 ✓
keycomputer92.com Surfshark
transactional Cybersecurity — 46 ✓
pencomputer92.com Surfshark
transactional Cybersecurity — 46 ✓
tabledoor92.com Surfshark
transactional Cybersecurity — 45 ✓
oncozine.com Onco'Zine: International Oncology Network \- Cancer \& Hematology News
informational Media Healthcare US 46 ✓
appesteem.com AppEsteem
interface Cybersecurity — 54 ✓ ✓
agentsroom.dev AgentsRoom
documentation AI and machine learning AI agents and MCP +1 — 50 ✓ ✓
easynac.com EasyNAC
documentation Infrastructure Cybersecurity +1 — 46 ✓
fluxlabs.net Flux Labs
documentation Cybersecurity — 51 ✓ ✓
swcp.com Southwest Cyberport
documentation Infrastructure US 57 ✓
ukcelebrities.co.uk UK Celebrities
informational GB 53 ✓
ivg-info.nl IVG info: Vitaminen, mineralen \& kruiden
documentation Healthcare NL 48 ✓
crawlora.net Crawlora
interface Real estate Ecommerce +3 — 50 ✓ ✓
japan-secure.com ネットセキュリティブログ
informational Cybersecurity JP 47 ✓
nightfall.ai Nightfall AI - The Agentic Data Loss Prevention Platform
documentation AI and machine learning Cybersecurity US 60 ✓
emeraldchat.com Emerald Chat
documentation Community — 57 ✓
mokkapps.de Michael Hoffmann - Senior Frontend Developer (Freelancer)
documentation Developer tools Personal sites and blogs DE 50 ✓ ✓
bitdefender.com Bitdefender | Global Leader in Cybersecurity
transactional Cybersecurity — 67 ✓
geminicli.com Gemini CLI Documentation
documentation AI and machine learning Developer tools US 57 ✓
centurylibrary.com Century Library - Vintage Design Assets for Commercial Use
transactional Design — 51 ✓ ✓
pixelplex.io PixelPlex
documentation Developer tools Finance and payments +4 — 53 ✓
homesavers.ie Homesavers Ireland
informational Marketing IE 47 ✓
partner.co Partner.co
transactional Fashion and beauty US 52 ✓
scrape.do Scrape.do
interface APIs and SDKs DO 51 ✓
blackpearlbitcoin.com BlackPearlBitcoin
informational Crypto and Web3 — 59 ✓
trusona.com Trusona
documentation Cybersecurity Authentication and identity — 49 ✓
rrecord.com Rappahannock Record: Serving the Northern Neck of Virginia
informational Marketing Media US 46 ✓
loopexdigital.com Loopex Digital
documentation Marketing US 61 ✓ ✓
tispy.net TiSPY
transactional — 56 ✓ ✓
valimail.com Valimail
documentation Marketing Cybersecurity US 58 ✓
xcloud.host xCloud
interface AI and machine learning Infrastructure +2 US 63 ✓ ✓
sianimage.com Sian Image Media — tipografie și producție publicitară (Constanța, România)
transactional Media Marketing +1 RO 58 ✓ ✓
nationalabortionfederation.org National Abortion Federation: NAF unites, supports, and represents abortion providers and the people they serve, in pursuit of accessible and equitable abortion care\.
informational Healthcare Nonprofit and advocacy — 58 ✓
vet-china.com Graphite Bipolar Plate，Graphite Ring/Bearing, Silicon Carbide Wafer Boat, Silicon Carbide Tube
informational Automotive Industry and manufacturing US 47 ✓
visitprijedor.com Turistička organizacija grada Prijedora
informational Travel SE 47 ✓
nodemaven.com NodeMaven
interface Infrastructure APIs and SDKs — 60 ✓
cointraffic.com Cointraffic
documentation Marketing Crypto and Web3 — 51 ✓ ✓
nbjge.com Amber Reflectors, Red Reflectors, Round Reflectors - J-GUANG
documentation Industry and manufacturing US 46 ✓
easydns.com easyDNS
transactional Infrastructure CA 65 ✓ ✓
borro.com Borro
documentation Arts and culture Finance and payments US 52 ✓ ✓
malwarebytes.com Malwarebytes
transactional Cybersecurity US 63 ✓
sunprotectionclothing.co.uk Sun Protection Clothing
transactional Fashion and beauty GB 57 ✓ ✓
qwaterfilters.com Φίλτρα Νερού QWaterFilters: Γιατί το καθαρό νερό είναι δικαιώμα όλων\.
transactional Ecommerce DE 53 ✓
armeriatomei.it Armeria Tomei
transactional Marketing Local business IT 47 ✓
drbackup.net Dr.Backup Data Protection Services
documentation Cybersecurity Infrastructure US 46 ✓
knockoff.co Knockoff
transactional Cybersecurity Ecommerce — 50 ✓
tauschspindel.com Tauschspindeln vom Spindeldoctor
transactional Industry and manufacturing DE 47 ✓
fortezzabellinzona.ch Fortezza Bellinzona
informational Arts and culture CH 46 ✓
globallegalinsights.com GLI
informational Legal GB 59 ✓
creationsmxfactory.fr Creations Mx Factory
transactional Automotive Ecommerce FR 50 ✓ ✓
armcokenya.com Armco Kenya: Make your house, a home\.\.\.
transactional Ecommerce DE 46 ✓
molekula.com Molekula
transactional Industry and manufacturing Science and research — 46 ✓
herbspro.com HerbsPro
transactional Ecommerce CA 61 ✓ ✓
redswitches.com RedSwitches
transactional Infrastructure US 57 ✓
lafilm.edu The Los Angeles Film School
informational Education US 48 ✓
immortals.com Immortals
transactional Healthcare Ecommerce CA 45 ✓ ✓
ud.hk UD (UDomain) — ud.hk
documentation Cybersecurity Infrastructure +1 HK 46 ✓
els.id ELS.ID
transactional Ecommerce Consumer electronics ID 52 ✓
getwoohoo.com WooHoo
documentation Ecommerce Marketing — 54 ✓
kaspersky.ca Kaspersky | Global Cybersecurity & Digital Privacy Solutions
transactional Cybersecurity CA 46 ✓
vibebot.gg VibeBot
transactional AI agents and MCP Developer tools +1 — 51 ✓ ✓
gunterwilhelm.com Günter Wilhelm® Kitchen Knives & Cookware
documentation — 49 ✓
secuodsoft.com Secuodsoft llms.txt
documentation Cybersecurity Business software IN 47 ✓
drip-apex.com Apex by DRIP
documentation Ecommerce Marketing +1 US 53 ✓ ✓
craigkeener.com Bible Backgrounds
documentation Marketing Personal sites and blogs US 46 ✓
arcturusrx.com Arcturus Therapeutics: Building Next Generation RNA Medicines
informational Healthcare — 51 ✓
phishprotection.com Phish Protection
transactional Cybersecurity US 56 ✓ ✓
data443.com Data443 — Real-Time AI Threat Intelligence & Data Security Platform
documentation Cybersecurity AI and machine learning US 48 ✓
tricksupply.com Tricksupply
transactional Marketing Ecommerce — 50 ✓ ✓
nouvellevie.com Nouvelle Vie
informational Community US 46 ✓
flyingbisons.com Flying Bisons
documentation Ecommerce Data and analytics +1 — 51 ✓ ✓
consegicbusinessintelligence.com Consegic Business Intelligence | Global Market Research Reports & Consulting
documentation Data and analytics — 48 ✓
alomone.com Alomone Labs
interface Science and research — 55 ✓
ladasten.com Ladasten — High-Purity Chemical Reagents for Laboratory Research
transactional Ecommerce Science and research PL 53 ✓ ✓
df.media Daily Finance
informational Marketing Media IN 45 ✓ ✓
fintecharbor.com Fintech Harbor Consulting
documentation Finance and payments Legal — 48 ✓
homeyohmy.com Homey Oh My
informational Personal sites and blogs Home and garden — 47 ✓
sicilianicreativiincucina.it Siciliani creativi in cucina: Blog di ricette di cucina e viaggi
documentation Food and dining IT 47 ✓
proginter.com Proginter
transactional Infrastructure IL 52 ✓ ✓
2-spyware.com 2-Spyware
documentation Cybersecurity — 57 ✓
csvlombardia.it CSV Lombardia
documentation Community IT 48 ✓
premiumpress.com PremiumPress
transactional Recruiting and careers Ecommerce CA 50 ✓
incuitypharma.com Incuity Pharma
transactional Healthcare ID 47 ✓
pollar.news Pollar
documentation Media — 50 ✓ ✓
iris-go.com Iris Web Framework
documentation Developer tools APIs and SDKs +1 US 47 ✓
audubonart.com Oppenheimer Gallery: The Natural History Art Gallery
transactional Arts and culture Ecommerce — 50 ✓
cranecpe.com CRANE ChemPharma \& Energy: ChemPharma \& Energy
documentation Industry and manufacturing US 56 ✓
afrotechmods.com Afrotechmods - Fun with electronics!
informational Consumer electronics Education US 48 ✓
actlegal.com actlegal
informational Legal SK 48 ✓
seamlessproxies.com seamless
transactional Infrastructure US 46 ✓
kaspersky.se Kaspersky | Globala cybersäkerhets- och digitala integritetslösningar
transactional Cybersecurity SE 47 ✓
developgoodhabits.com Develop Good Habits
informational Personal sites and blogs — 60 ✓
berkeleysprings.com America's First Spa - Berkeley Springs, West Virginia
documentation Travel US 47 ✓
cdgkz-relay.com CDGKZ
documentation Industry and manufacturing LT 51 ✓
advancedceramics.co.uk Discover Premium Ceramic Products | Durability & Elegance United | Advanced Ceramic
documentation GB 45 ✓
b2binpay.com B2BINPAY
interface Finance and payments Crypto and Web3 — 57 ✓
zugferdpro.com ZUGFeRD Pro
documentation Developer tools APIs and SDKs ES 45 ✓ ✓
faros.ai Faros AI
documentation Developer tools AI and machine learning +1 US 58 ✓
hototools.com HOTO
transactional Home and garden Consumer electronics CA 54 ✓ ✓
mediagraming.com Mediagraming | Navigating Finance, Savings, and Retirement with Confidence
documentation Finance and payments Healthcare +1 — 45 ✓
payneteasy.com Payneteasy
documentation Finance and payments — 50 ✓ ✓
warwickartscentre.co.uk Warwick Arts Centre
informational Arts and culture Entertainment and streaming GB 61 ✓ ✓
ginernet.com GINERNET
transactional Infrastructure ES 62 ✓ ✓
see.news SEE News
informational Media — 58 ✓
shaka.deal Shaka (shaka.deal)
interface Finance and payments Crypto and Web3 +1 — 57 ✓
bip39-phrase.com BIP39 Phrase
documentation Crypto and Web3 Documentation and reference — 51 ✓ ✓
blockaid.io https://blockaid.io llms-full.txt
documentation Cybersecurity Crypto and Web3 — 51 ✓
masspointpllc.com MassPoint Legal PLLC
informational Legal US 47 ✓
ladyran.com لیدیران: فروش آنلاین محصولات آرایشی بهداشتی
transactional Fashion and beauty Ecommerce IR 46 ✓
meridianbioscience.com Meridian Bioscience
informational Marketing Healthcare +1 US 52 ✓ ✓
prosopo.io Prosopo
documentation Cybersecurity NL 55 ✓ ✓
nationalsalonsupplies.com.au National Salon Supplies: Wholesale Salon \& Beauty Supplies
transactional Ecommerce Fashion and beauty AU 48 ✓
interpeace.org Interpeace: Enabling people to build lasting peace since 1994
informational Marketing Nonprofit and advocacy US 48 ✓
rarebreedtriggers.com Rare Breed Triggers
transactional Marketing Ecommerce — 54 ✓
hairmd.com.tr HairMD Clinical Series Products
transactional Marketing Healthcare +1 TR 47 ✓
proxidize.com Proxidize
interface Infrastructure APIs and SDKs — 62 ✓ ✓
itamarnovick.com Insights, not just posts
informational Community Personal sites and blogs US 46 ✓
ajtractorparts.com AJ Tractor Parts USA | Buy Restoration Parts | Buy Farming Equipment and Antique Tractor Parts.
transactional Agriculture Automotive +1 US 47 ✓
bitcoinworld.co.in BitcoinWorld
informational Finance and payments Crypto and Web3 +1 IN 55 ✓
zhhimg.com Precision Ceramic Solutions, Precision Metal Solutions, Precision Glass Solutions - INTELLIGENT
transactional Industry and manufacturing US 52 ✓
knowledgeofhealth.com Knowledge of Health
informational US 47 ✓
techjaja.com Techjaja
informational AI and machine learning Finance and payments +1 FR 47 ✓
ipfighter.com IPFighter - Check your IP score | Advanced IP fraud detection
informational Cybersecurity — 53 ✓