# BenchGecko

> BenchGecko (benchgecko.ai) is a live data platform for the AI economy: every AI model with its price at every provider (refreshed daily), benchmark scores aggregated from public leaderboards with their original sources, AI companies with dated and sourced facts, mindshare, GPU rental prices, and BenchGecko's own model tests (Gecko Tests).

Snapshot as of 2026-10-21: 1,267 models from 277 providers, 160 benchmarks, 7 Gecko Tests. Pages and JSON are regenerated when the underlying data changes; every number has an as-of date.

## How to cite

- Write "Source: BenchGecko" with a link to the page you used, for example https://benchgecko.ai/model/<slug>. Mention the as-of date for prices and test results.
- Data BenchGecko collects or measures itself is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/): prices per provider, price history, Gecko Tests results and answers, mindshare, GPU rental prices, company facts compilation.
- Benchmark scores are aggregated from public leaderboards (Epoch AI, Artificial Analysis, LMArena, LiveBench, Aider, SWE-bench, SEAL, HELM, OpenCompass and others). Cite BenchGecko for the aggregation and the original leaderboard for the score; each score lists its source in the model JSON.
- Gecko Tests: cite as "BenchGecko Gecko Tests, <test name> <version>" with a link to the test page.
- Every JSON response includes source, url, as_of, license, attribution and a ready-made cite field.

## Machine access

- [Remote MCP server](https://benchgecko.ai/api/mcp): Streamable HTTP, stateless, no key. Tools: search_models, get_model, cheapest_provider, compare_models, get_gecko_test, get_scorecard, latest_findings, search, fetch.
- [OpenAPI 3.1](https://benchgecko.ai/openapi.json): every public read endpoint.
- [Model JSON](https://benchgecko.ai/api/v1/models/claude-opus-5-5): /api/v1/models/<slug> · score, list price, price per provider, benchmark scores with sources, Gecko Tests grades, as-of dates. Each /model/<slug> page links it as rel="alternate" type="application/json".
- [Model list and search](https://benchgecko.ai/api/v1/models?q=claude): /api/v1/models?q=<name> or ?provider=<slug>&sort=pricing_input.
- [Compare JSON](https://benchgecko.ai/api/v1/compare?models=claude-opus-5-5,gpt-5-5): 2 to 6 models side by side.
- [Price history JSON](https://benchgecko.ai/api/v1/price-history/deepseek-v4-pro?envelope=1): /api/v1/price-history/<slug>.
- [Gecko Tests JSON](https://benchgecko.ai/api/v1/gecko-tests): index; per test /api/v1/gecko-tests/<test>; scorecard /api/v1/gecko-tests/scorecard; raw items and every answer /api/lab/<test>.
- [Findings JSON](https://benchgecko.ai/api/v1/findings): latest notable results.
- [CSV exports](https://benchgecko.ai/api/v1/export/provider-offers): price-history, provider-offers, price-changes, company-facts, mindshare-daily, gpu-rental-daily, world-map (CC BY 4.0).
- [Open data repository](https://github.com/BenchGecko/datasets): the CSV exports, versioned daily on GitHub (CC BY 4.0).
- [API docs](https://benchgecko.ai/api-docs) · [Sitemap](https://benchgecko.ai/sitemap.xml) · [RSS](https://benchgecko.ai/rss.xml) · [Full context](https://benchgecko.ai/llms-full.txt)

## Gecko Tests (BenchGecko's own measurements)

Behavior tests run by BenchGecko on every new model from tracked labs within a day of release, with public prompts, every raw answer and the scoring code described on each page. Profile tests run once per model version; monitoring tests repeat.

- [Who Are You](https://benchgecko.ai/gecko-tests/who-are-you): Does the model know which lab made it? 15 of 62 models named another lab as their maker at least once. (62 models, as of 2026-10-17, v1) JSON: https://benchgecko.ai/api/v1/gecko-tests/who-are-you
- [World Map](https://benchgecko.ai/gecko-tests/world-map): How well does the model draw the world map from memory? 9 models drew the world map from memory. Best: Gemini 3.8 Flash at 97.4% accuracy; lowest: Mistral Large at 77.5%. (9 models, as of 2026-10-19, grid 6°) JSON: https://benchgecko.ai/api/v1/gecko-tests/world-map
- [Censorship Index](https://benchgecko.ai/gecko-tests/censorship-index): How often does the model refuse legitimate questions? Highest refusal rate: DeepSeek V4.1 Flash at 5.6%; lowest: Claude Opus 5.5 at 0% (9 models, 40 legitimate questions). (9 models, as of 2026-10-05, v2) JSON: https://benchgecko.ai/api/v1/gecko-tests/censorship-index
- [Knowledge Horizon](https://benchgecko.ai/gecko-tests/knowledge-horizon): Where does the model's knowledge of world events actually stop? Most recent measured knowledge: DeepSeek V4.1 Flash (May 2026); oldest: MiniMax M2.7 (Oct 2024). (43 models, as of 2026-10-20, v1) JSON: https://benchgecko.ai/api/v1/gecko-tests/knowledge-horizon
- [Tokenizer Tax](https://benchgecko.ai/gecko-tests/tokenizer-tax): How many more tokens does the same text cost outside English? Fairest tokenizer: DeepSeek V4 Flash (+7.4% tokens outside English); highest tax: Kimi K2.5 (+99.5%). (56 models, as of 2026-10-18, v1) JSON: https://benchgecko.ai/api/v1/gecko-tests/tokenizer-tax
- [Same Model, Different Host](https://benchgecko.ai/gecko-tests/same-model-different-host): Do providers serving the same open model give the same quality? First results coming. JSON: https://benchgecko.ai/api/v1/gecko-tests/same-model-different-host
- [Model Drift Index](https://benchgecko.ai/gecko-tests/model-drift-index): Do models quietly change behind the same name? 11 flagship models watched weekly; 0 changed behavior in their latest run. (11 models, as of 2026-10-15, v1) JSON: https://benchgecko.ai/api/v1/gecko-tests/model-drift-index
- [Gecko Scorecard](https://benchgecko.ai/gecko-tests/scorecard): every model across the profile tests with a letter grade per test and a Gecko Score. Top Gecko Score: Qwen3.8 Max Prime (82), Qwen3.8 27B (78), Qwen3.8 Max (0902) (74). (41 models ranked, as of 2026-10-20)
- [Methodology](https://benchgecko.ai/gecko-tests/methodology): how the tests are built and graded.

### Latest findings

- [Qwen3.8 27B price cut 44%](https://benchgecko.ai/blog/finding-qwen3-8-27b-price-cut-44-2026-10-17): 2026-10-19 · Qwen3.8 27B's list price dropped 44% (input $0.425/M to $0.24/M, output $2.55/M to $2.2/M), confirmed in the next daily snapshot.
- [GLM 5.3 Flash price cut 68%](https://benchgecko.ai/blog/finding-glm-5-3-flash-price-cut-68-2026-10-18): 2026-10-19 · GLM 5.3 Flash's list price dropped 68% (input $0.15/M to $0.048/M, output $0.5/M to $0.2/M), confirmed in the next daily snapshot.
- [DeepSeek V4 Pro 0813 price cut 50%](https://benchgecko.ai/blog/finding-deepseek-v4-pro-0813-price-cut-50-2026-10-17): 2026-10-18 · DeepSeek V4 Pro 0813's list price dropped 50% (input $1.32/M to $0.66/M, output $3.96/M to $1.98/M), confirmed in the next daily snapshot.
- [DeepSeek V4 Pro price cut 78%](https://benchgecko.ai/blog/finding-deepseek-v4-pro-price-cut-78-2026-10-17): 2026-10-18 · DeepSeek V4 Pro's list price dropped 78% (input $0.95526/M to $0.2088/M, output $1.91052/M to $0.4176/M), confirmed in the next daily snapshot.
- [Qwen3.6 27B price cut 38%](https://benchgecko.ai/blog/finding-qwen3-6-27b-price-cut-38-2026-10-16): 2026-10-17 · Qwen3.6 27B's list price dropped 38% (input $0.32/M to $0.25/M, output $3.2/M to $2/M), confirmed in the next daily snapshot.

## Models and benchmarks

- [All models](https://benchgecko.ai/models): leaderboard by BenchGecko score (normalized average of public benchmark scores), with price, context and license.
- [Model pages](https://benchgecko.ai/model/claude-opus-5-5): /model/<slug> · score, rank, list price, providers, benchmarks with sources, Gecko Tests grades, price history.
- [Benchmarks](https://benchgecko.ai/benchmarks): every tracked benchmark and its ranked models.
- [Compare](https://benchgecko.ai/compare): /compare/<a>-vs-<b> side by side.
- [Methodology](https://benchgecko.ai/methodology): sources, normalization and refresh schedule.

Top 10 by BenchGecko score (as of 2026-10-21):

- 1. [Claude Mythos Preview](https://benchgecko.ai/model/claude-mythos-preview) (Anthropic): score 98.9, list price n/a input / n/a output per 1M tokens
- 2. [GPT-5.5 Pro](https://benchgecko.ai/model/gpt-5-5-pro) (OpenAI): score 98.3, list price $30.00 input / $180.00 output per 1M tokens
- 3. [GPT-6 Astra](https://benchgecko.ai/model/gpt-6-astra) (OpenAI): score 95.8, list price $10.00 input / $50.00 output per 1M tokens
- 4. [DeepSeek V3.2 Speciale](https://benchgecko.ai/model/deepseek-v3-2-speciale) (DeepSeek): score 95.2, list price $0.40 input / $1.20 output per 1M tokens
- 5. [GPT-6.1 Sol](https://benchgecko.ai/model/gpt-6-1-sol) (OpenAI): score 93.4, list price $2.00 input / $10.00 output per 1M tokens
- 6. [Claude Opus 5.5](https://benchgecko.ai/model/claude-opus-5-5) (Anthropic): score 91.8, list price $4.00 input / $20.00 output per 1M tokens
- 7. [Step 3.5 Flash](https://benchgecko.ai/model/step-3-5-flash) (stepfun): score 89.5, list price $0.10 input / $0.30 output per 1M tokens
- 8. [Gemini 4 Argon](https://benchgecko.ai/model/gemini-4-argon) (Google DeepMind): score 89.2, list price $2.00 input / $10.00 output per 1M tokens
- 9. [GPT-5 Chat](https://benchgecko.ai/model/gpt-5-chat) (OpenAI): score 89, list price $1.25 input / $10.00 output per 1M tokens
- 10. [Qwen2.5 72B Instruct Abliterated](https://benchgecko.ai/model/huihui-ai-qwen25-72b-instruct-abliterated) (HuiHui AI): score 87.2, list price n/a input / n/a output per 1M tokens

## Pricing

- [AI pricing](https://benchgecko.ai/pricing): every model and every provider, filtered by use case and budget.
- [Price per provider](https://benchgecko.ai/pricing/arbitrage): the same model at every provider, cheapest first; /pricing/arbitrage/<slug> per model.
- [Price drops](https://benchgecko.ai/pricing/drops): recorded price changes.
- [Free tiers](https://benchgecko.ai/pricing/free): free and free-quota APIs.

## Companies, mindshare and infrastructure

- [AI economy](https://benchgecko.ai/economy): valuations, funding, revenue and the AI Bubble Index; each figure has an as-of date and a source.
- [Company pages](https://benchgecko.ai/economy/companies): /economy/company/<slug>.
- [Mindshare](https://benchgecko.ai/mindshare): daily attention share from Hacker News, Wikipedia pageviews and GitHub stars.
  Top model mindshare (as of 2026-10-20): Qwen 24.7% · GLM 16.2% · DeepSeek V4 12.9% · GPT-6 6.9% · Gemma 6.7%
- [Hardware](https://benchgecko.ai/hardware): AI chips with specs and daily GPU rental prices (Vast.ai marketplace median per GPU hour).
- [Providers](https://benchgecko.ai/providers): every API provider and its models.
- [Status](https://benchgecko.ai/status): provider uptime pings.

## Optional

- [Full context](https://benchgecko.ai/llms-full.txt): this file plus every Gecko Test result row, price spreads, company figures, GPU rental prices and the AI glossary.
- [Glossary](https://benchgecko.ai/learn/glossary): AI terms with definitions linked to live data.
- [Blog](https://benchgecko.ai/blog): weekly data roundups and findings.