# Viteval

> TypeScript-native evals, powered by Vitest.

## Features

- [Features](https://viteval.dev/features.md): Native Vitest workflows, built-in scoring, local results, and evaluations on your own infrastructure.

## Guide

- [Getting started](https://viteval.dev/guide/getting-started.md): Run a passing evaluation, save its JSON report, and inspect it locally. Requires Node >=26. [!NOTE] V1 is currently a release candidate. Install viteval@rc; untagged installs still resolve to v0. V0 support ends when v1 launches.

## API

- [evaluate()](https://viteval.dev/api/evaluate.md): evaluate() creates a Vitest suite with a test for each case. It runs your task, scores the output, and checks the threshold.

## Examples

- [Examples](https://viteval.dev/examples.md): These examples cover deterministic checks, generated datasets, and model-based scoring. The first two run without an API key.

## Others

- [API reference](https://viteval.dev/api.md)
- [Configuration](https://viteval.dev/api/configuration.md): Configure Viteval in vitest.config.ts using Vitest's test options. Initialize models and storage providers in a worker setup file.
- [createScorer()](https://viteval.dev/api/create-scorer.md): createScorer() gives a scoring function a name. Use the returned scorer in an evaluation's scorers array.
- [defineDataset()](https://viteval.dev/api/define-dataset.md): defineDataset() wraps a case generator in a reusable dataset. Pass the dataset to evaluate() or save it with viteval data.
- [scorers](https://viteval.dev/api/scorers.md): Viteval includes deterministic scorers, embedding similarity scorers, and LLM judges. Call each factory to create a scorer. LLM and embedding scorers use the models initialized in Vitest setup. See Configuration. The scores in the examples illustrate how to read results. Model-based scores vary with the configured model and are not guarantees of correctness or safety. Judge scorers accept { model } to override the initialized language model. Model failures return score: null with an error in metadata; null scores contribute 0 to evaluation aggregation.
- [Brand & assets](https://viteval.dev/brand.md): Viteval logos, icons, motion, colors, and typography. Download the assets for your next integration.
- [Guide](https://viteval.dev/guide.md)
- [Advanced](https://viteval.dev/guide/advanced.md)
- [CI integration](https://viteval.dev/guide/advanced/ci.md): Run viteval run in CI to check prompt and application changes against your evaluation suite. A failed case returns a nonzero exit code.
- [Reporters](https://viteval.dev/guide/advanced/reporters.md): Use Vitest reporters for test results and Viteval's JSON reporter for evaluation inputs, outputs, and scores.
- [CLI usage](https://viteval.dev/guide/cli.md): Use the CLI to initialize projects, run evaluations, generate datasets, and open the dashboard. Evaluation commands pass through to Vitest.
- [Core concepts](https://viteval.dev/guide/concepts.md): An evaluation runs a task against test cases and scores each output. Datasets hold the cases; thresholds determine whether they pass.
- [Migrate to v1](https://viteval.dev/guide/migration.md): V1 requires Node >=26 and uses native Vitest 5 configuration.
- [Providers](https://viteval.dev/guide/providers.md): Viteval is open source and uses local JSON storage by default. You do not need a hosted evaluation account. With the default JSON provider, datasets live under .viteval/datasets and runs under .viteval/results. The JSON reporter also writes local snapshots for dashboard inspection. Configured hosted providers such as Braintrust, Langfuse, and VoltOps persist runs through their own services; they do not move those hosted runs into the local directory. @viteval/providers and every backend SDK/driver are opt-in. Importing a provider factory does not load its SDK. Install the packages for the backend you select; initialization errors list any missing packages and how to install them. Use @viteval/providers@rc alongside viteval@rc during the prerelease. The untagged provider package can resolve to an older release candidate. Choose providers independently for datasets and evaluation results: These providers store evaluation data. They do not choose the model used by your task. Call your model client inside the task function; configure models for built-in judges and embedding scorers with initializeModel().