# RubyLLM Documentation

> Developer documentation for RubyLLM 2.1.1 (stable).

The canonical docs cover installation, configuration, providers, core features, agents, evaluations, observability, Rails integration, and the complete reference. Also available: the [API reference Markdown index](https://rubyllm.com/api/index.md), the live [model registry as JSON](https://rubyllm.com/models.json), the [source](https://github.com/crmne/ruby_llm), and the [RubyGems package](https://rubygems.org/gems/ruby_llm).

## Full Documentation

- [Complete documentation](https://rubyllm.com/llms-full.txt): All canonical pages in one Markdown document.

## Overview

- [RubyLLM](https://rubyllm.com/): Build and evaluate AI features in Ruby and Rails with chats, tools, agents, images, audio, and video across 19 providers.

## Start Here

- [Getting Started](https://rubyllm.com/getting-started/): Install RubyLLM and build with chats, tools, agents, images, video, audio, and document processing in Ruby and Rails.
- [How RubyLLM Works](https://rubyllm.com/overview/): Understand RubyLLM's public API, its providers and protocols, and the services they share.
- [Configuration](https://rubyllm.com/configuration/): Set provider credentials, choose default models, and share configuration across the RubyLLM API.
- [Provider Setup and Custom Endpoints](https://rubyllm.com/configuration-providers/): Configure provider credentials and connect to hosted or local APIs through compatible endpoints.
- [Connection, Logging and Contexts](https://rubyllm.com/configuration-connection/): Timeouts, retries, proxies, connection reuse, logging, the model registry file, and isolated multi-tenant contexts.
- [What's New in 2.1](https://rubyllm.com/whats-new-in-2-1/): Connect to MCP servers, ask typed judgments, evaluate your agents, and trace them with OpenTelemetry in RubyLLM 2.1.
- [What's New in 2.0](https://rubyllm.com/whats-new-in-2-0/): Explore the expanded provider coverage, new AI operations, conversation controls, and Rails integration in RubyLLM 2.0.
- [AI Coding Assistants](https://rubyllm.com/ai-coding-assistants/): Install the RubyLLM skill to help your coding assistant use the API and documentation that match your application.

## Core Features

- [Chat](https://rubyllm.com/chat/): Start a conversation, give it instructions, and switch models while keeping the same Ruby API.
- [Attachments](https://rubyllm.com/attachments/): Ask questions about images, recordings, videos, and documents through one attachment API.
- [Stream Responses](https://rubyllm.com/streaming/): Learn how to display AI responses in real-time as they're generated
- [Structured Output](https://rubyllm.com/structured-output/): Get AI responses that match an exact JSON schema with required fields and types
- [Extended Thinking](https://rubyllm.com/thinking/): Give reasoning models more time and budget to deliberate, with optional access to thinking output
- [Citations](https://rubyllm.com/citations/): Get verifiable answers with normalized citations pointing at documents and web sources, on every provider that supports them
- [Prompt Rendering](https://rubyllm.com/prompt-rendering/): Render reusable ERB prompt templates from app/prompts with RubyLLM.render_prompt
- [Prompt Caching](https://rubyllm.com/prompt-caching/): Reuse stable prompt prefixes with automatic or explicit prompt caching
- [Advanced Request Control](https://rubyllm.com/chat-request-control/): Reach provider-specific features with custom parameters, wire protocols, request hooks, and HTTP headers
- [Tokens and Costs](https://rubyllm.com/cost-and-usage-tracking/): Read normalized token counts and costs for every response, chat, and provider attempt, including retries and cancellations.
- [Chat Event Handlers](https://rubyllm.com/chat-callbacks/): Hook into the chat lifecycle with additive callbacks for UI updates, logging, and analytics
- [Tools](https://rubyllm.com/tools/): Let AI call your Ruby code. Connect to databases, APIs, or any external system with function calling.
- [Tool Parameters](https://rubyllm.com/tool-parameters/): Declare tool arguments - from inferred signatures to full JSON Schema, rich return values, and provider-specific metadata.
- [Controlling Tool Execution](https://rubyllm.com/tool-execution/): Choose tools, require approval, run calls concurrently, and observe their results
- [MCP Client](https://rubyllm.com/mcp/): Connect your chats and agents to Model Context Protocol servers and use their tools, resources, and prompts
- [Provider Tools](https://rubyllm.com/provider-tools/): Search the web, run code, and connect remote MCP tools through the same chat API
- [Tool Search](https://rubyllm.com/tool-search/): Give a model hundreds of tools without sending them all, and let it search for the ones it needs.
- [Embeddings](https://rubyllm.com/embeddings/): Create vectors from text, images, audio, video, and documents for search and similarity
- [Reranking](https://rubyllm.com/rerank/): Order candidate documents by how well they answer a query, the second stage of a retrieval pipeline
- [Image Generation](https://rubyllm.com/image-generation/): Generate and edit images from text prompts, reference images, and masks
- [Judgments](https://rubyllm.com/judgments/): Ask typed questions about your application data and receive probabilities, choices, and scores
- [Video Generation](https://rubyllm.com/video-generation/): Generate, edit, and extend videos from prompts, images, and audio
- [Audio Transcription](https://rubyllm.com/audio-transcription/): Convert speech to text with support for multiple languages and speaker diarization
- [Text to Speech](https://rubyllm.com/text-to-speech/): Convert text into spoken audio
- [Document OCR](https://rubyllm.com/ocr/): Extract text, tables, and images from PDFs and scans as clean markdown
- [Moderation](https://rubyllm.com/moderation/): Screen text and images for harmful content before it reaches your models
- [Files](https://rubyllm.com/files/): Upload files once, reuse them in chats and batches, and download provider results.
- [Tokenization](https://rubyllm.com/tokenization/): Inspect token IDs and count a conversation's input tokens before generating a response.

## Building Real Apps

- [Rails Integration](https://rubyllm.com/rails/): Use the RubyLLM API with Active Record, Active Storage, Hotwire, and your existing Rails jobs.
- [Persistence with acts_as](https://rubyllm.com/rails-persistence/): Persist application-owned chats and messages while RubyLLM manages its supporting records.
- [Streaming with Hotwire/Turbo](https://rubyllm.com/rails-streaming/): Broadcast AI responses token by token to the browser with Turbo Streams and background jobs.
- [Generators and App Conventions](https://rubyllm.com/rails-generators/): Scaffold chats, a chat UI, agents, tools, and schemas with RubyLLM's Rails generators and conventions.
- [Advanced Rails Configuration](https://rubyllm.com/rails-advanced-config/): Route models through different providers, use per-tenant contexts, persist cache boundaries, adjust provider payloads per request, and run fiber-safe.
- [Agents](https://rubyllm.com/agents/): Give agents instructions, tools, and schemas in a Ruby class, then use them in scripts, services, and Rails jobs.
- [Agentic Workflows](https://rubyllm.com/agentic-workflows/): Compose agents and AI operations with ordinary Ruby methods, from sequential steps to routing and parallel work.
- [Retrieval-Augmented Generation (RAG)](https://rubyllm.com/rag/): Retrieve relevant context from your own documents, then answer with that context
- [Durable Agents](https://rubyllm.com/durable-agents/): Run agent turns as background jobs that survive crashes, deploys, and the wait for a human decision.
- [Memory](https://rubyllm.com/memory/): Give agents short-term memory through the transcript and long-term memory your app owns, recalled by meaning.
- [Batches](https://rubyllm.com/batches/): Process chats and embeddings in batches, then collect their results when they are ready.
- [Scale with Async](https://rubyllm.com/async/): Run AI jobs with Solid Queue fiber workers, use Async for concurrent Ruby calls, and choose Async::Job for higher throughput.
- [Instrumentation and Observability](https://rubyllm.com/instrumentation/): Observe RubyLLM model operations, provider attempts, tool calls, workflows, batches, and model refreshes
- [OpenTelemetry](https://rubyllm.com/opentelemetry/): Trace RubyLLM model calls, tools, and workflows with your application's OpenTelemetry setup
- [Error Handling](https://rubyllm.com/error-handling/): Rescue provider errors, fall back to other models, and let automatic retries absorb transient failures
- [Hosted Research](https://rubyllm.com/hosted-research/): Run a hosted research agent with remote tools and collect its report through a durable job ID
- [Evaluations](https://rubyllm.com/evaluations/): Measure how well your agents answer, with reusable datasets, model grading, and Ruby assertions
- [Evaluation Datasets](https://rubyllm.com/evaluation-datasets/): Define evaluation cases in YAML, JSON, JSONL, or application code
- [Evaluating Conversations and Tools](https://rubyllm.com/evaluation-conversations/): Evaluate multi-turn conversations, saved transcripts, tool calls, and your own result objects
- [Evaluators](https://rubyllm.com/evaluation-evaluators/): Decide what a good answer means and which model, Agent, or Judge assesses it
- [Running Evaluations](https://rubyllm.com/evaluation-running/): Run evaluations from Rake, RSpec, Minitest, or Ruby and read their results, tokens, and costs
- [Evaluation Progress and Monitoring](https://rubyllm.com/evaluation-progress/): Show evaluation progress in your application, save trials as they finish, and trace runs
- [Evaluation Examples](https://rubyllm.com/evaluation-examples/): Complete evaluations for formatting, grounded answers, and safe tool use that you can adapt

## Reference

- [API Reference](https://rubyllm.com/api-reference/): RDoc documentation for every public RubyLLM class and method
- [Model Registry](https://rubyllm.com/models/): Find models by provider or capability, inspect their pricing, and refresh the registry in plain Ruby and Rails.
- [Model Resolution](https://rubyllm.com/model-resolution/): How RubyLLM turns a model name into a concrete model and provider, step by step, covering aliases, the registry, provider preference, and unlisted models.
- [Custom Endpoints and Unlisted Models](https://rubyllm.com/custom-endpoints/): Target OpenAI-compatible endpoints and use model IDs the registry doesn't list
- [Custom Providers and Protocols](https://rubyllm.com/custom-providers/): Teach RubyLLM to talk to a new AI service by writing a provider, a protocol, or both, and ship it as a gem.
- [Upgrading](https://rubyllm.com/upgrading/): Move a RubyLLM 2.0 application to 2.1, one release at a time.
- [RubyLLM Ecosystem](https://rubyllm.com/ecosystem/): Extend RubyLLM with structured schemas, instrumentation, monitoring and community-built tools for production AI apps.

## Pages

- [About RubyLLM](https://rubyllm.com/about/): RubyLLM is an open source Ruby gem for building AI applications across major AI providers with one consistent Ruby framework.