iris-eval.com valid

Iris — stop shipping agents on vibes

Iris is an open-source MCP server that scores AI agent output quality, catches safety failures (PII leaks, prompt injection, hallucination markers), and enforces cost budgets. Any MCP client lists its tools on connect — one command (`npx -y @iris-eval/mcp-server@0.20.0`), no SDK; your agent calls them, a host hook forwards each turn, or `iris-eval ingest` loads a trace file. MIT-licensed core, self-hosted, signed releases.

For agents: Open-source MCP server that scores AI agent output quality, flags PII leaks, prompt injection and hallucinations, and enforces cost budgets.

mcp serveragent evaluationllm output qualityprompt injection detectionpii leak detectioncost budgets

Their llms.txt

The beginning of the file, from our validated copy saved 2026-10-24T10:14:09.759Z. Cached copy · Source on iris-eval.com

# Iris — stop shipping agents on vibes

> Iris is an open-source MCP server that scores AI agent output quality, catches safety failures (PII leaks, prompt injection, hallucination markers), and enforces cost budgets. Any MCP client lists its tools on connect — one command (`npx -y @iris-eval/mcp-server@0.20.0`), no SDK; your agent calls them, a host hook forwards each turn, or `iris-eval ingest` loads a trace file. MIT-licensed core, self-hosted, signed releases.

Current release: v0.20.0 (2026-10-03) — Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges.

Iris registers 12 MCP tools (trace logging, heuristic eval, LLM-as-judge, semantic citation verification, rule lifecycle) and ships 26 built-in eval rules across completeness, relevance, safety, and cost. 10 of those rules read the agent's own tool calls and judge what it DID, not only what it wrote: a failed call the answer never acknowledges, a repeated one, a call its own tool's schema rejects, a file or URL cited that appears in nothing the agent read, an instruction obeyed out of a tool result, and a step budget. A ninth custom rule type, action_policy, states which tools an agent may call and with what arguments. All of it is deterministic and runs locally — no model call, nothing leaves the machine. It is verified on every CI run in Claude Code, and Gemini CLI; for Claude Desktop, Cursor, Devin Desktop (Windsurf), Continue, VS Code, Cline, Zed, and OpenAI Codex CLI the installer writes the configuration shape each client documents and that writer is tested on the shape, with nobody on the Iris side having watched the client connect (claimed — every row with its source and date at https://iris-eval.com/clients); and it runs in any other MCP client the same way.

Evaluator accuracy is published at https://iris-eval.com/proof: precision, recall and F1 with 95% confidence intervals for built-in rules — 13 measured for detection against labels a model gave by reading the failure (synthetic, model-labelled corpus; a human blind label is pending), and 13 checked against their own documented formula (a score there shows the code implements the formula, not that it detects the failure) — corpus 7c27824a2b98, generated 2026-10-04 from the source these numbers were measured on; reproduce with `npm run proof`.

Of 60 capability cells (10 evaluation questions by 6 subjects), 34 are answered by a shipped, measured thing, 18 are answered with a stated limit, 4 are open gaps and 4 do not apply — every answered cell names the rule, tool, resource, route, proof row or judge template behind it. The full map: https://iris-eval.com/capabilities.

## Getting Started

- [GitHub repository](https://github.com/iris-eval/mcp-server): source, README with per-client setup for every major MCP client, full tool schemas
- Claude Code plugin: `/plugin marketplace add iris-eval/mcp-server` then `/plugin i
…

Related sites

Explore similar publishers →

Similar publisher descriptions and documentation. Similarity does not imply affiliation or equivalent capabilities.

Machine-readable: GET /v1/resolve?q=iris-eval.com · GET /v1/domain/iris-eval.com · this page as Accept: application/json or text/markdown · MCP lookup_domain

Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.

Claim my listing on Agent Web Map for iris-eval.com:
1. POST https://agentwebmap.com/v1/sites/iris-eval.com/claims and keep claim_id and claim_secret private.
2. Publish the proof: add the line "AgentWebMap ownership: <token>" (token from step 1) to https://iris-eval.com/llms.txt, or use the DNS, meta-tag or well-known method the response describes.
3. POST https://agentwebmap.com/v1/sites/iris-eval.com/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/iris-eval.com).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verify

Corrections or removal requests: contact the directory.