---
title: Jetty
description: Grade agent turns, compare experiments, and store durable evaluation trajectories.
type: extension
keywords:
  - evals
  - evaluation
  - grading
  - experiments
  - observability
  - trajectories
  - bandit
  - a/b testing
---

# Jetty

Extension integration for eve. Grade agent turns, compare experiments, and store durable evaluation trajectories.

## Install

Install the Jetty extension for eve:

```bash
eve add extension/jetty
```

The extension requires Node.js 24 or later and eve 0.25 or later. It can ingest every completed turn as a durable Jetty trajectory, grade turns inline, steer experiments from their grades, and report native `eve eval` results.

## Quick start

Add your Jetty credentials and collection to the agent's environment:

```bash title=".env.local"
JETTY_API_TOKEN=your_token
JETTY_COLLECTION=your_collection
```

Then mount the extension under `agent/extensions/`:

```ts title="agent/extensions/jetty.ts"
import jetty from "@jetty/eve";

export default jetty({
  collection: process.env.JETTY_COLLECTION ?? "",
  task: "triage-live",
  judgeMode: "simple_judge",
  arms: {
    warm: "Write a warm, specific response.",
    terse: "Write a concise, direct response.",
  },
});
```

The filename supplies the `jetty` namespace. The extension contributes a turn-ingestion hook, dynamic instructions that select an experiment arm, and `jetty__experiment`, which reports per-arm results and the current leader. Create the `simple_judge` task in Jetty before using inline grading; use the default `ingest` mode when a separate grader will score trajectories later.

## Configure

The package also includes a reporter for eve's native eval runner:

```ts title="evals/evals.config.ts"
import { Jetty } from "@jetty/eve/reporter";
import { defineEvalConfig } from "eve/evals";

export default defineEvalConfig({
  reporters: [Jetty()],
});
```

The reporter reads `JETTY_API_TOKEN` and `JETTY_COLLECTION`, sends each eval result to Jetty, and warns rather than failing the eval when Jetty is unavailable. The extension no-ops when its collection is empty, so the same agent can run without Jetty credentials.

Jetty trajectories persist agent inputs and outputs. Redact PII before grading, put sensitive grader parameters in Jetty's `secretParams` rather than `initParams`, and treat trajectory storage like any other logging surface. See the [Jetty eve extension documentation](https://github.com/jettyio/jetty-sdk/tree/main/packages/eve#readme) for all experiment settings and the [worked example](https://github.com/jettyio/jetty-sdk/tree/main/examples/eve-jetty) for the complete grading loop.

[Read the full extension documentation](https://github.com/jettyio/jetty-sdk/tree/main/packages/eve#readme)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)