eriehome.com Erie Home LLM Access Policy
informational US 46 ✓
myrobotdoll.com My Robot Doll
documentation 52 ✓ ✓
slido.com Slido
interface Communications Productivity 62 ✓
thewaternetwork.com The Water Network | by AquaSPE
informational Community DE 49 ✓
robotvacuummops.uk Robot Vacuum Mops
transactional Ecommerce Home and garden GB 61 ✓ ✓
cdnetworks.com 规则组 1: 针对所有爬虫的通用规则
documentation FR 48 ✓
nerdvpn.de nerdvpn.de and all subdomains
informational AI and machine learning DE 45 ✓
imoo.com imoo Global Site Index
transactional Consumer electronics US 55 ✓
getlatka.com GetLatka
documentation Data and analytics 63 ✓ ✓
diligentrobots.com Diligent Robotics
documentation Healthcare US 49 ✓
leesa.com Leesa
informational US 48 ✓
learn-atc.com Learn ATC
transactional Education DE 50 ✓ ✓
progresschamber.org Chamber of Progress
informational Nonprofit and advocacy 47 ✓
trimble.com Corporate
informational CA 64 ✓
mlslistings.com MLSListings
documentation Real estate US 57 ✓
howthelightgetsin.org howthelightgetsin.org
documentation GB 46 ✓
bretton.com Bretton
documentation US 45 ✓
corvus-robotics.com Corvus Robotics
documentation AI and machine learning Industry and manufacturing +1 DE 47 ✓
aps-bom.com Authorized Procurement Solutions (APS)
documentation Industry and manufacturing 47 ✓
freehost.com.ua Разрешаем OpenAI GPTBot (ChatGPT)
documentation UA 50 ✓
robotimeonline.com LLMs — www.robotimeonline.com
documentation AI and machine learning CA 49 ✓ ✓
maticrobots.com Matic Robots
transactional Consumer electronics 56 ✓
runway.com Runway
documentation Developer tools US 65 ✓
uengage.io uEngage
transactional Marketing Ecommerce +1 48 ✓
saramin.co.kr 사람인 (Saramin)
interface Recruiting and careers AI agents and MCP KR 57 ✓
robotframework.org Robot Framework
documentation Open source projects Developer tools US 54 ✓ ✓
cactus.md Cactus.md
transactional Ecommerce Consumer electronics 51 ✓ ✓
thetradehub.eu The Trade Hub
documentation Business software 47 ✓
carclean.com Carclean
transactional AI and machine learning Ecommerce NL 49 ✓
bedrockrobotics.com Bedrock Robotics
documentation Industry and manufacturing Construction and architecture US 47 ✓
news1.kr News1 (뉴스1)
informational Media KR 58 ✓
checkbox.ai Checkbox
documentation Legal Business software US 46 ✓
refero.design Refero
interface AI and machine learning Design +1 57 ✓
kasada.io Kasada
interface Cybersecurity US 49 ✓
fieldly.com LLMs.txt for Fieldly (Sverige)
documentation AI and machine learning Construction and architecture +1 US 48 ✓
bytepe.com BytePe
transactional Ecommerce Consumer electronics US 51 ✓ ✓
chipoteka.hr Chipoteka
transactional Ecommerce Consumer electronics HR 50 ✓
hashmicro.com HashMicro
documentation AI and machine learning Business software 57 ✓
michaelzingraf.com Michael Zingraf Real Estate
documentation Real estate Local business FR 53 ✓ ✓
sunbeltrentals.com Sunbelt Rentals LLMs.txt
documentation 58 ✓
cipherhealth.com CipherHealth
documentation Healthcare Business software US 48 ✓
omegapediatrics.com Omega Pediatrics
transactional AI and machine learning Healthcare 51 ✓ ✓
bitcoinwisdom.io BitcoinWisdom
documentation Crypto and Web3 Data and analytics 50 ✓
tuition.io Tuition.io
documentation Recruiting and careers US 45 ✓
openclaw.ai OpenClaw
documentation AI and machine learning AI agents and MCP US 47 ✓
vectoral.com Vectoral
documentation AI and machine learning Cybersecurity US 46 ✓
stripchat.red stripchat.red
documentation 51 ✓ ✓
bothunt.ru BotHunt
documentation Cybersecurity RU 56 ✓ ✓
toolai.io ToolAI.io
informational AI and machine learning HK 49 ✓
poeoverlay.com PoE Overlay
transactional Gaming 56 ✓
peakvisor.com PeakVisor
documentation Sports and fitness Travel DE 59 ✓
slidy.ai Slidy AI
documentation AI and machine learning RU 51 ✓
smakprov.se Smakprov.se
informational Media SE 50 ✓
microsiervos.com llms.txt - AI Training Policy for Microsiervos.com
informational Media 54 ✓
whisperweb.dev Whisper Web
transactional Media AI and machine learning 50 ✓ ✓
toolsense.io ToolSense
documentation Business software US 49 ✓
davidlahey.com David Lahey
documentation 48 ✓
lifeofpoker.com LifeOfPoker
informational Sports and fitness Media SG 47 ✓
knownagents.com Known Agents
interface AI and machine learning AI agents and MCP +1 US 57 ✓
everygen.ai Everygen
documentation AI and machine learning US 45 ✓ ✓
superpowerdaily.com Superpower Daily
informational AI and machine learning Media 49 ✓ ✓
orthotw.com Orthopedics This Week
informational Healthcare Media US 48 ✓
quakemap24.com QuakeMap24
documentation Science and research DE 47 ✓
glicerink.com Glice AG
transactional Industry and manufacturing Sports and fitness 46 ✓
huayan-robotics.net Huayan Robotics
documentation Industry and manufacturing US 46 ✓
wakool.games Wakool.Games
informational Gaming 46 ✓
hungama.com Hungama.com LLM Access Policy
informational Entertainment and streaming Media IN 52 ✓
zone.eu Zone.eu
interface EE 56 ✓
corneliajames.com Cornelia James
transactional Fashion and beauty US 47 ✓
doctormgt.com DoctorMGT
documentation Healthcare US 48 ✓
nohacks.co No Hacks - AI Agent Guidelines
informational AI and machine learning Media +1 51 ✓ ✓
cable-sleeving.com MDPC-X
transactional AI and machine learning Ecommerce DE 46 ✓
litt.ly Littly (리틀리)
documentation Marketing KR 58 ✓
ieee-ras.org IEEE Robotics and Automation Society Website: Focus is on both applied and theoretical issues in robotics and automation\. Robotics is here defined to include intelligent machines and systems; whereas
documentation Marketing Science and research US 59 ✓
platinum-international.store Platinum International Co. for Industrial Technology
transactional Industry and manufacturing Ecommerce 55 ✓
schulstart.de Schulstart
transactional Ecommerce AI and machine learning DE 46 ✓
bresser.de Bresser
transactional AI and machine learning Ecommerce +1 DE 52 ✓
skins.nl Skins
transactional AI and machine learning Ecommerce NL 52 ✓
smprobotics.com SMP Robotics
documentation Industry and manufacturing AI and machine learning 47 ✓
camelactive.com camelactive.com
transactional AI and machine learning Ecommerce 51 ✓
spin.ai Spin.AI
documentation AI and machine learning Cybersecurity +1 51 ✓
wirwinzer.de wirwinzer.de
transactional AI and machine learning Ecommerce DE 51 ✓
hepco-becker.de hepco-becker.de
transactional AI and machine learning Ecommerce +1 DE 50 ✓
koffer24.de Koffer24
transactional AI and machine learning Ecommerce DE 50 ✓
bresser.com Bresser
transactional AI and machine learning Ecommerce +1 DE 49 ✓
mediatel.at media.tel
transactional AI and machine learning Ecommerce AT 49 ✓
iai.tv IAI.tv
informational GB 55 ✓
tauchversand.com Divestore
transactional AI and machine learning Ecommerce DE 49 ✓
fortune.nl fortunecoffee
transactional AI and machine learning Ecommerce +1 NL 48 ✓
leiste24.de Leiste24
transactional AI and machine learning Ecommerce DE 48 ✓
bresser.fr Bresser
transactional AI and machine learning Ecommerce +1 FR 47 ✓
demmer-shop.de Demmer
transactional AI and machine learning Ecommerce DE 47 ✓
noch.com NOCH
transactional AI and machine learning Ecommerce DE 47 ✓
bresserbenelux.nl Bresser
transactional AI and machine learning Ecommerce +1 NL 46 ✓
bresser.it Bresser
transactional AI and machine learning Ecommerce +1 IT 46 ✓
einlagen-shop.com einlagen-shop.com
transactional AI and machine learning Ecommerce DE 46 ✓
mykitchens.de Mykitchens
documentation AI and machine learning Ecommerce +1 DE 46 ✓
solardirekt24.de Solardirekt24
transactional AI and machine learning Ecommerce +1 DE 46 ✓
cvlid.com Cvlid
documentation Recruiting and careers 46 ✓
solar-haus.de solar-haus.de
transactional AI and machine learning Ecommerce +1 DE 46 ✓
higuchi.com higuchi\.com 2\.0: The Means Justifies the Ends
informational Marketing Personal sites and blogs 46 ✓
kmong.com Kmong
transactional JP 57 ✓
devalk.nl De Valk Yacht Brokers
informational Ecommerce NL 46 ✓
besserdampfen.de besserdampfen
transactional AI and machine learning Ecommerce DE 45 ✓
zoundhouse.de zoundhouse.de
transactional AI and machine learning Ecommerce DE 47 ✓
fischersports.com Fischer Sports
transactional AI and machine learning Ecommerce +1 DE 59 ✓
raab-verlag.de Raab Verlag
transactional AI and machine learning Ecommerce DE 49 ✓
pfundskerl-xxl.de Pfundskerl Onlineshop
transactional AI and machine learning Ecommerce DE 48 ✓
salesrobot.co SalesRobot
transactional Marketing Business software US 49 ✓
centinelanalytica.com Centinel Analytica
documentation 58 ✓ ✓
apksundays.com Apksundays
informational Marketing Entertainment and streaming FR 47 ✓
hibt.com HiBT Exchange
informational 50 ✓
dirjournal.com DirJournal: The Global Business Trust Graph
documentation US 49 ✓ ✓
vertu.com VERTU
transactional Consumer electronics CA 60 ✓
webotit.ai Webotit.ai
documentation AI and machine learning US 58 ✓ ✓
manbiz.gr manbiz
documentation GR 47 ✓
openvibe.bot OpenVibe.Bot
documentation Developer tools 45 ✓ ✓
trainocate.co.jp トレノケート株式会社 LLMs.txtポリシー / Trainocate Corporation AI Content Policy (LLMs.txt) | This file defines AI crawling and training permissions for LLMs. Please read carefully. | LLMs.txt Policy for AI Crawler
informational JP 47 ✓
nerdytips.com NerdyTips
documentation AI and machine learning Sports and fitness 61 ✓ ✓
flexibleassembly.com Flexible Assembly Systems
transactional Industry and manufacturing Ecommerce US 46 ✓
vistarmedia.com LLMs.txt for Vistar Media (vistarmedia.com)
documentation AI and machine learning Media +1 51 ✓
gomechanic.in llms.txt for gomechanic.in
documentation IN 47 ✓
51papaya.com 51PAPAYA
documentation Entertainment and streaming AI and machine learning US 52 ✓ ✓
puzl.com AI Access Policy for https://www.Puzl.com
US 46 ✓
buffget.com Buffget
transactional Gaming Ecommerce 48 ✓
yamakei-online.com AIクローラーによる要約と引用の許可設定
informational JP 49 ✓
duckietown.com Duckietown
documentation Education 51 ✓ ✓
smmgen.com SMMGen
transactional Marketing SG 49 ✓
bubblealba.com 버블알바
documentation Recruiting and careers Community 49 ✓
typhur.com Typhur Robotics (https://www.typhur.com/)
transactional Home and garden Consumer electronics CA 49 ✓
romimo.ro Romimo
informational Real estate RO 47 ✓
milremrobotics.com Milrem
documentation Industry and manufacturing 45 ✓
evwire.com EVwire
informational Media Automotive 48 ✓
taass.com TAASS.com
documentation AI and machine learning Ecommerce DE 48 ✓
wsu-industrials.com WSU INDUSTRIALS
transactional AI and machine learning Ecommerce DE 46 ✓
gardeur.com gardeur Deutschland
transactional AI and machine learning Ecommerce +1 DE 46 ✓
carepilot.com CarePilot
documentation Healthcare AI and machine learning US 46 ✓
sweco.fi Sweco Finland
documentation Construction and architecture FI 47 ✓
wazuh.com Wazuh
documentation Cybersecurity 62 ✓
phoebe.work Phoebe
transactional Healthcare AI and machine learning +1 US 46 ✓
mattayaclinic.com Mattaya Vision Center
transactional Local business Healthcare 46 ✓
ewaahotels.com EWAA Hotels
transactional Travel US 47 ✓
techexplorist.com Tech Explorist: Source for Latest Technology \& Science News
informational Media Science and research 54 ✓
sweco.se Sweco Sweden
documentation Construction and architecture SE 52 ✓
swecogroup.com Sweco Group
informational Construction and architecture SE 51 ✓
sweco.nl Sweco Nederland
informational Construction and architecture NL 51 ✓
swecobelgium.be Sweco Belgium
documentation Construction and architecture BE 48 ✓
sweco.dk Sweco Denmark
documentation Construction and architecture DK 47 ✓
franz-mensch.de Franz Mensch
transactional AI and machine learning Ecommerce DE 47 ✓
akkushop.de AkkuShop-Deutschland
transactional AI and machine learning Ecommerce DE 51 ✓
seit1887.de Kräuterparadies seit 1887
transactional AI and machine learning Ecommerce DE 47 ✓
walhalla.de WALHALLA Fachverlag
transactional Healthcare Government and public services +1 DE 47 ✓
carpocratian.org The Carpocratian Church & School
documentation Nonprofit and advocacy US 52 ✓
quoka.de Quoka
informational Real estate Ecommerce DE 53 ✓
laendleanzeiger.at Laendleanzeiger
informational Real estate Ecommerce AT 48 ✓
riskyjatt.is RiskyJatt.Com
documentation Media Entertainment and streaming 51 ✓ ✓
axial.net Axial - LLM Orientation File
documentation Finance and payments Business software US 47 ✓
emeritus.org Emeritus
documentation Education 60 ✓
workable.com Workable | All-in-one HR software
documentation Recruiting and careers Business software 66 ✓
sweco.co.uk Sweco United Kingdom
documentation Construction and architecture GB 48 ✓
cemnet.com CemNet
informational Media Industry and manufacturing 52 ✓
technisat.de TechniSat Online-Shop
transactional Ecommerce AI and machine learning +1 DE 53 ✓
deutschesee.de Deutsche See B2C (DE)
transactional AI and machine learning Ecommerce DE 51 ✓
velken.de Velken-Folientechnik DE
transactional AI and machine learning Ecommerce DE 47 ✓
it-harvest.com IT-Harvest
documentation Cybersecurity Data and analytics 47 ✓
startapro.hu Startapro
informational Real estate Community HU 48 ✓
romjob.ro Romjob
informational Recruiting and careers RO 47 ✓
bestauto.ro Bestauto
informational Automotive RO 46 ✓
animalutul.ro Animalutul
informational Pets RO 45 ✓
electronicsexpo.com Electronics Expo
transactional Ecommerce Consumer electronics CA 51 ✓ ✓
starnewskorea.com StarNews Korea
informational Media Entertainment and streaming 60 ✓
plazapremiumlounge.com Plaza Premium Lounge
transactional AI and machine learning Travel 56 ✓ ✓
thecouchcritic.com The Couch Critic
documentation Entertainment and streaming 48 ✓
theonespy.com TheOneSpy
documentation 51 ✓
longsight.com Longsight
documentation Education US 46 ✓
softbankrobotics.com SoftBank Robotics Group Corp.
informational Marketing Industry and manufacturing 60 ✓
salmoiraghievigano.it Salmoiraghi & Viganò
transactional Ecommerce Fashion and beauty IT 45 ✓
fruit.com Fruit of the Loom
transactional Fashion and beauty US 53 ✓
r34.app Rule 34 App
informational 55 ✓
gaugedb.com GaugeDB
documentation Gaming Data and analytics 47 ✓
djooni.com Djooni
documentation Media Entertainment and streaming 47 ✓
dm-folien.com dm-folien - Ihr Partner für Folien
transactional AI and machine learning Ecommerce DE 47 ✓
create-learn.us Create & Learn
interface Education US 55 ✓ ✓
minotaurus.io Minotaurus
transactional Crypto and Web3 46 ✓
baitukas.lt Baitukas.lt
transactional Gaming Ecommerce LT 48 ✓
otypo.com Otypo — otypo.com
transactional Ecommerce 48 ✓
mk.co.kr Maeil Business Newspaper
documentation Media KR 64 ✓ ✓
andlight.com AndLight
transactional Home and garden 55 ✓
therundown.ai The Rundown AI
informational AI and machine learning Media +1 US 59 ✓
gorbatech.com GorbaTech Solutions
documentation Infrastructure Marketing GR 50 ✓ ✓
riskyjatt.com RiskyJatt.Com
documentation Media Entertainment and streaming 51 ✓ ✓
pogodnik.com Pogodnik — Ukrainian Weather Forecast Service
informational Media 46 ✓
par71.de Par71.de
transactional Ecommerce AI and machine learning DE 47 ✓
impulsecreative.com Impulse Creative
documentation AI and machine learning Business software +1 US 54 ✓ ✓
informdirect.co.uk Inform Direct
documentation Business software GB 57 ✓
garmtech.com GARMTECH
transactional Infrastructure GB 58 ✓
insightbase.com InsightBase
documentation Data and analytics 50 ✓
cycleai.ai Cycle AI
informational AI and machine learning Entertainment and streaming 55 ✓
gui.do Guido X Jansen — gui.do
documentation Community DO 52 ✓ ✓
xoxostickers.com XOXO Stickers
documentation 47 ✓
logoi.health Logoi
interface Science and research Documentation and reference 51 ✓ ✓
coinlib.io Coinlib
interface Data and analytics Crypto and Web3 US 55 ✓
9mothers.com Starter
documentation US 45 ✓
solverify.net Solverify
interface APIs and SDKs 52 ✓
vbotickets.com VBO Tickets
documentation Arts and culture Marketing +2 56 ✓
dirox.com Dirox
documentation Business software SG 47 ✓
hellosays.com HelloSays
informational Local business 58 ✓ ✓
ip-lookup.net IP-Lookup.net
informational Infrastructure 46 ✓
itecs.ai ITECS AI
documentation AI and machine learning Cybersecurity US 45 ✓ ✓
recordstoreday.co.uk Record Store Day UK
informational Entertainment and streaming GB 53 ✓
statewise.com Statewise
documentation Finance and payments Healthcare +1 US 48 ✓
scoutbasketball.com Scoutbasketball
documentation Sports and fitness Data and analytics DE 47 ✓
akprinthub.com AkPrintHub
documentation Productivity 47 ✓
manuals.plus Manuals+
documentation Documentation and reference 67 ✓ ✓
awwwards.com Awwwards
interface Design NL 63 ✓
gardena.com GARDENA
documentation Home and garden NL 62 ✓
website.com Website.com
transactional Ecommerce Infrastructure +2 62 ✓ ✓
elephantstock.com Elephant Stock
transactional Home and garden Arts and culture CA 61 ✓ ✓
hyperdash.com Hyperdash
documentation Crypto and Web3 Data and analytics US 61 ✓ ✓
uic.org UIC — International Union of Railways
documentation Industry and manufacturing US 60 ✓
addictioncenter.com Addiction Center
informational Healthcare 57 ✓
brownbook.net Brownbook – Global Business Listing Database
interface Data and analytics 57 ✓
evkx.net EVKX.net
documentation Media Automotive NO 57 ✓
expertflyer.com ExpertFlyer
documentation Travel 57 ✓
fifplay.com FIFPlay
documentation Gaming 57 ✓
itjobswatch.co.uk IT Jobs Watch
documentation Recruiting and careers Data and analytics GB 57 ✓
webhostingtalk.nl Webhostingtalk.nl
informational Infrastructure Community NL 57 ✓
zalify.com Zalify
interface Marketing US 57 ✓
e-inscricao.com E-inscricao
transactional Events and conferences 56 ✓
ijr.com IJR.com
informational Media Community 56 ✓
tapnow.ai TapNow
documentation AI and machine learning US 55 ✓
fontainebleau.com Fontainebleau
transactional Travel 54 ✓
customuse.com Customuse
interface Design AI and machine learning DE 53 ✓ ✓
djjohal.com DJJohal
informational Entertainment and streaming Media 53 ✓
sociallykeeda.com SociallyKeeda
informational Media 53 ✓ ✓
claimavenue.com Claim Avenue
transactional 52 ✓ ✓
getthematic.com Thematic
documentation AI and machine learning Business software 52 ✓
hobby-caravan.de Hobby Caravan
informational Automotive DE 52 ✓
myrecipia.com MyRecipia
documentation Food and dining 52 ✓
nukeworker.com NukeWorker.com
documentation Recruiting and careers Community +1 US 52 ✓ ✓
paitolottosdy.info PaitoLottoSDY
informational Media 52 ✓ ✓
printpal.io PrintPal
interface AI and machine learning APIs and SDKs +1 52 ✓ ✓
studiestimer.com StudiesTimer
transactional Productivity Education US 52 ✓ ✓
betteam.pro Betteam.pro — Sports Media, Match Predictions & Betting Education
informational Education Media +2 RU 51 ✓
huarenca.com Huarenca
informational Community Recruiting and careers 51 ✓
jobiba.com Jobiba
interface Recruiting and careers 51 ✓ ✓
mattresswarehouse.com Mattress Warehouse
transactional Ecommerce Home and garden CA 51 ✓
onewaytour.com ONE WAY TOUR — Licensed travel agency in Yerevan, Armenia
transactional Travel 51 ✓ ✓
rynda.io Rynda — live vessel tracking, alerts and vessel checks
interface Logistics and shipping DE 51 ✓ ✓
seomodi.com SEOmodi
interface Marketing TR 51 ✓ ✓