aievals.co valid
AI Evals
The most useful, opinionated reference for AI evals.
- Authority
- 45 reach 0 · trust 5 · quality 25 · freshness 15
- Tier
- listed
- llms.txt
- https://www.aievals.co/llms.txt
- llms-full.txt
- https://www.aievals.co/llms-full.txt
- Sections
- 13
- Links
- 1134
- Size
- 155034 bytes
- Country
- — Publisher location unknown; CDN address excluded
- Country evidence
- No published address evidence collected yet
- Categories
- AI and machine learning
- For agents
- Documentation: developer or product docs worth reading, nothing to call
- First seen
- 2026-10-27T11:35:57.037Z
- Last checked
- 2026-10-27T11:35:59.985Z
- Badge
Their llms.txt
The beginning of the file, from our validated copy saved 2026-10-27T11:35:57.044Z. Cached copy · Source on aievals.co
# AI Evals
> The most useful, opinionated reference for AI evals.
## About this resource
AI Evals is a curated technical reference on evaluating large language model systems in production. The site covers the methodology (error analysis, LLM-as-judge, statistical rigor), the tools (Langfuse, Phoenix, Promptfoo, Inspect, RAGAS), the canonical papers, and twenty-six task-specific eval playbooks. Every claim is cited. Content is licensed CC BY 4.0; please cite https://www.aievals.co when you reference material from here.
Last generated: 2026-10-16T23:42:56.203Z.
Author: Om Bharatiya (https://www.linkedin.com/in/ombharatiya/).
## Top entry points
- [Start Here for each persona](https://www.aievals.co/start): linear reading path per role.
- [Cookbook](https://www.aievals.co/cookbook): runnable worked examples in code.
- [The Glossary](https://www.aievals.co/glossary): definitions for every term used.
- [All papers](https://www.aievals.co/papers): ten themed reading lists.
- [All tools](https://www.aievals.co/tools): vendor and OSS positioning.
- [Free PDF downloads](https://www.aievals.co/downloads): every track and section as a PDF field guide; free account required.
## Most cited pages on this site
> Pages with the highest citation density. Useful for answer-engine grounding.
- [Reasoning depth and thinking-mode analysis](https://www.aievals.co/task-types/reasoning-depth) (27 citations, Task-type playbooks): Evaluate the thinking traces of reasoning models. Correctness on hard reasoning plus chain-of-thought faithfulness, not just answer accuracy.
- [LLM-as-Judge](https://www.aievals.co/techniques/llm-as-judge) (23 citations, Techniques): The comprehensive reference: what an LLM judge is, when it works, the four-part prompt, calibration discipline, the three biases, distillation, where judges fail.
- [Reward hacking detection](https://www.aievals.co/learn/agentic-evals/reward-hacking) (20 citations, Agentic Evals): Once an objective is optimized, it gets gamed. Reward hacking is empirically common; eval for it explicitly.
- [Long-context RAG and RULER](https://www.aievals.co/learn/rag-evals/long-context-rag) (19 citations, RAG Evals): Why advertised context lengths exceed effective context lengths, and the benchmarks that expose the gap.
- [Where to keep your eval datasets](https://www.aievals.co/learn/datasets/dataset-platforms) (19 citations, Datasets & Golden Sets): Langfuse, Phoenix, Braintrust, LangSmith, or a JSONL in git. The choice that actually matters.
- [Autonomous coding](https://www.aievals.co/task-types/autonomous-coding) (18 citations, Task-type playbooks): Given an issue and a repo, autonomously edit files until hidden tests pass. Execution-grounded scoring is the bar, Verified is saturated, SWE-bench Pro is harder but has broken tasks of its own, and...
- [MCP and tool-registry evals](https://www.aievals.co/task-types/mcp-tool-evals) (18 citations, Task-type playbooks): Evaluating agents on MCP servers: MCP-Universe, tool-r
…Related sites
Similar publisher descriptions and documentation. Similarity does not imply affiliation or equivalent capabilities.
| Domain | Title | Category | Country | Authority | llms.txt | llms-full.txt | Save |
|---|---|---|---|---|---|---|---|
| deepchecks.com | Deepchecks | documentation AI and machine learning DevOps and observability | US | 56 | ✓ | ||
| deepeval.com | Docs | documentation AI and machine learning Developer tools | US | 52 | ✓ | ✓ | |
| alloevil.github.io | agent-harness-evals | documentation AI and machine learning Developer tools | US | 45 | ✓ | ✓ | |
| ragas.io | Ragas | documentation AI and machine learning Developer tools | — | 57 | ✓ | ✓ | |
| vals.ai | Vals AI | documentation AI and machine learning | US | 64 | ✓ | ✓ | |
| vladyslavpodoliako.com | Vlad's Playbook — The Ultimate AI Dive Deep | documentation AI and machine learning Documentation and reference | US | 50 | ✓ | ✓ | |
| teammately.ai | Teammately Docs | interface AI agents and MCP Documentation and reference | — | 50 | ✓ | ✓ | |
| qfloo.com | QFLOO | documentation Education AI and machine learning | US | 50 | ✓ | ✓ |
Machine-readable: GET /v1/resolve?q=aievals.co · GET /v1/domain/aievals.co · this page as Accept: application/json or text/markdown · MCP lookup_domain
Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.
Own or manage this website?
If you own or manage aievals.co, you can update its listing and review file health, discovery activity and score history.
Manage this listing — optional
What verification unlocks
See how your site is discovered—and control what agents see.
- See your website in the map
Review your indexed topics, nearby publishers sharing headings, measured listing exposure and map snapshots in a private report for your website.
- Control what agents see
Edit your listing description, tags, public contact details and additional llms.txt paths. Publish your terms for automated access, attribution and pricing.
- Make your profile eligible for search
A valid, listed, unblocked, verified profile can appear in our publisher sitemap and become eligible for search-engine visibility. Search engines decide what to index and rank.
- Appear in our llms-full.txt
Eligible verified publishers are listed by name, with their file links and summaries, in the service overview agents can read.
Free. No account registration, backlink or payment required. Public discovery remains available without verification.
Let your coding agent verify it
If your coding agent is authorized to manage this website, paste these instructions into it.
Claim my listing on Agent Web Map for aievals.co:
1. POST https://agentwebmap.com/v1/sites/aievals.co/claims and keep claim_id and claim_secret private.
2. Publish the verification line "AgentWebMap verification: <token>" (token from step 1) in https://aievals.co/llms.txt, or use the DNS, meta-tag or well-known method the response describes. This only verifies the listing; your website and domain remain yours.
3. POST https://agentwebmap.com/v1/sites/aievals.co/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/aievals.co).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verifyCorrections or removal requests: contact the directory.