rlhfbook.com valid

RLHF Book

A free online book and course on reinforcement learning from human feedback (RLHF), reward models, preference tuning, RLVR, and post-training language models.

For agents: Free online book and course on RLHF, reward models, preference tuning, RLVR and post-training language models, with full-text markdown and a model comparison li

rlhfreward modelspreference tuningpost-training llmsdpo

Their llms.txt

The beginning of the file, from our validated copy saved 2026-10-21T18:23:03.674Z. Cached copy · Source on rlhfbook.com

# RLHF Book

> A free online book and course on reinforcement learning from human feedback (RLHF), reward models, preference tuning, RLVR, and post-training language models.

The RLHF Book by Nathan Lambert explains how modern language models are post-trained, from instruction tuning and preference data to reward modeling, policy optimization, direct alignment, reasoning training, evaluation, and product behavior.

Use the chapter links for canonical web pages. Use `llms-full.txt` for a single concatenated Markdown context file generated from the public chapter sources.

## Core Resources

- [Full-text LLM context](https://rlhfbook.com/llms-full.txt): Concatenated Markdown for all public book chapters, excluding the references build utility page.
- [Book homepage](https://rlhfbook.com/): The canonical web version of the RLHF Book.
- [Course](https://rlhfbook.com/course): Free lectures and course material on RLHF and post-training.
- [Model library](https://rlhfbook.com/library): Model completion comparisons across supervised finetuning, RLHF, DPO, and related post-training stages.
- [RL cheatsheet](https://rlhfbook.com/rl-cheatsheet): One-page reference for PPO, GRPO, RLOO, REINFORCE, DPO, and related RLHF methods.

## Chapters

- [Introduction](https://rlhfbook.com/c/01-introduction): A first-principles introduction to RLHF, what it changes in language models, and how it became part of modern post-training.
- [A Tiny History of RLHF](https://rlhfbook.com/c/02-related-works): Key papers and historical milestones behind RLHF, reward modeling, preference learning, and post-training language models.
- [Training Overview](https://rlhfbook.com/c/03-training-overview): A high-level map of modern post-training recipes, including instruction tuning, RLHF, RLVR, and direct alignment methods.
- [Instruction Fine-Tuning](https://rlhfbook.com/c/04-instruction-tuning): How instruction tuning turns base language models into usable assistants and sets up later RLHF and post-training stages.
- [Reward Modeling](https://rlhfbook.com/c/05-reward-models): How reward models are trained from preference data and used as the learned objective in RLHF post-training pipelines.
- [Reinforcement Learning](https://rlhfbook.com/c/06-policy-gradients): Policy gradient methods for RLHF and LLM post-training, including PPO, REINFORCE, RLOO, GRPO, and implementation details.
- [Reasoning and Inference-Time Scaling](https://rlhfbook.com/c/07-reasoning): Reasoning training and inference-time scaling in post-training, including RLVR and thinking models.
- [Direct-Alignment Algorithms](https://rlhfbook.com/c/08-direct-alignment): Direct alignment algorithms such as DPO that optimize preference objectives without an explicit reward model or RL loop.
- [Rejection Sampling](https://rlhfbook.com/c/09-rejection-sampling): Rejection sampling and best-of-n methods for improving post-trained language models with reward or preference signals.
- [The Nature of Preferences](https://rlhfbook.co
…

Related sites

Other listed sites in AI and machine learning. See the whole category →

Machine-readable: GET /v1/resolve?q=rlhfbook.com · GET /v1/domain/rlhfbook.com · this page as Accept: application/json or text/markdown · MCP lookup_domain

Automatically discovered from public files. Inclusion does not imply endorsement or publisher participation.

Claim my listing on Agent Web Map for rlhfbook.com:
1. POST https://agentwebmap.com/v1/sites/rlhfbook.com/claims and keep claim_id and claim_secret private.
2. Publish the proof: add the line "AgentWebMap ownership: <token>" (token from step 1) to https://rlhfbook.com/llms.txt, or use the DNS, meta-tag or well-known method the response describes.
3. POST https://agentwebmap.com/v1/sites/rlhfbook.com/verify with {"claim_id": "...", "claim_secret": "..."}.
4. Save the api_key from the answer; it manages the listing (PATCH https://agentwebmap.com/v1/sites/rlhfbook.com).
MCP equivalent: tools begin_ownership_claim, then verify_domain, at https://agentwebmap.com/mcp. Docs: https://agentwebmap.com/docs#verify

Corrections or removal requests: contact the directory.