> ## Documentation Index
> Fetch the complete documentation index at: https://rllm-org-rllm-19-feat-renderer-parser-backend.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Harnesses

> Built-in AgentFlows for common shapes of work — one-shot LLM, bash ReAct, Claude Code

A *harness* is a built-in [AgentFlow](/core-concepts/agentflow-evaluator): a generic agent program parameterised by the LLM and the task, with no per-benchmark code. rLLM ships three of them. The CLI's `--agent` flag picks one by name; the [Runner](/core-concepts/task-and-runner) drives it like any other AgentFlow.

You don't need a harness for everything — for one-off agents and cookbooks, hand-write an `AgentFlow` class. Reach for a harness when the same agent shape applies to many tasks (every math benchmark needs a one-shot LLM call; every SWE-bench-style task needs an interactive bash loop).

## Built-in harnesses

| Name          | Class                                          | Use case                                                                |
| ------------- | ---------------------------------------------- | ----------------------------------------------------------------------- |
| `react`       | `rllm.harnesses.react.ReActHarness`            | One-shot LLM call. Default for data tasks (math, MCQ, QA).              |
| `bash`        | `rllm.harnesses.bash.BashHarness`              | Multi-turn ReAct bash loop inside a sandbox. Default for sandbox tasks. |
| `claude-code` | `rllm.harnesses.claude_code.ClaudeCodeHarness` | Run the Claude Code CLI inside the sandbox.                             |

The registry lives at `rllm/registry/agents.json`. `load_agent("react")` resolves to `ReActHarness()`.

## ReAct harness — one-shot LLM

`ReActHarness` is the default for catalog datasets (gsm8k, MATH-500, MMLU, ...). It makes one chat completion and packs the response into a single-step Episode:

```python theme={null}
from rllm.harnesses.react import ReActHarness

agent = ReActHarness(system_prompt="You are a careful math tutor.")
episode = agent.run(task, config)
```

The harness automatically appends the verifier's expected output format to the system prompt — so a math benchmark gets `"Put your final answer in \boxed{}"` injected, while an MCQ benchmark gets `"Respond with a single letter A, B, C, or D"`. See [Reward functions](/core-concepts/reward-functions) for how that hint is declared.

`trajectory.output` is set to the LLM response so reward functions can extract the answer with a uniform contract.

## Bash harness — sandboxed ReAct loop

`BashHarness` is the default for sandbox tasks. It runs a multi-turn ReAct loop where the agent's tool is `bash` executed inside the task's sandbox. Each turn:

1. The model proposes a bash command (or signals it's done).
2. The harness runs it in the sandbox and returns stdout/stderr.
3. The conversation continues until the model declares completion or a turn cap is hit.

Use this for benchmarks where success is measured by the state of files in a sandboxed environment — code repairs, terminal tasks, environment configuration.

## Claude Code harness

`ClaudeCodeHarness` runs the Claude Code CLI inside the sandbox as a black-box agent. The harness is a thin wrapper that streams Claude Code's output and packages the resulting trajectory. Use it when the benchmark expects a fully agentic coding workflow and you want Claude Code's tool-use behaviour out of the box.

## Resolving a harness

The CLI resolves harnesses by name through `load_agent`:

```python theme={null}
from rllm.eval import load_agent

agent = load_agent("react")               # Built-in harness
agent = load_agent("my_pkg:my_agent")     # Module path:attribute
```

`load_agent` accepts:

* A registered name from `rllm/registry/agents.json` (`react`, `bash`, `claude-code`).
* A `module.path:attribute` reference for any AgentFlow you've installed.

`dataset.toml` can declare a `default_agent` that points at a harness name; `BenchmarkLoader` surfaces it on `BenchmarkResult.harness_name`, and the CLI uses it when `--agent` is not supplied.

## Writing a custom harness

A harness is just an AgentFlow with a class-level `name` and (optionally) `max_concurrent`:

```python theme={null}
from rllm.types import Episode, Step, Trajectory, Task


class MyHarness:
    name = "my-harness"
    max_concurrent = 32

    def __init__(self, system_prompt: str | None = None):
        self.system_prompt = system_prompt or "You are helpful."

    async def arun(self, task: Task, config) -> Episode:
        # ... call the model, build steps, etc.
        step = Step(input=task.instruction, output="...")
        traj = Trajectory(steps=[step], output=step.output)
        return Episode(trajectories=[traj])
```

Register it by adding an entry to `rllm/registry/agents.json` (for built-ins) or by referencing it as `my_pkg.my_module:MyHarness` from the CLI:

```bash theme={null}
rllm eval gsm8k --agent my_pkg.my_module:MyHarness
```

If your harness needs a sandbox, subclass `rllm.sandbox.SandboxedAgentFlow` instead — the Runner will provision a fresh sandbox per task.