Code Extension

Mount eve's coding extension, eve-code, and see its latest benchmark results against other coding agent harnesses.

eve/extensions/code (eve-code) turns an eve agent into a coding agent. It ships inside the eve package, so you mount it without installing anything else. The extension contributes file-editing and search tools, an authenticated GitHub CLI tool, coding skills, a read-only worker subagent, and sandbox bootstrap helpers.

Benchmark results

deepswe-lean@v1 · claude-sonnet-5.53 contenders across 2 runs

higher is better

0%25%50%75%100%

correctnessvs ref

pi@0.85.1

72%+6 pp

eve-code@4d14914

67%ref

opencode@1.18.30

50%−17 pp

Lines are 95% intervals; overlapping lines are not a measured difference. Click a contender to open its run.

These results show baseline performance. Each harness uses its standard setup with high reasoning. The results do not show how a harness performs with other models, settings, or tasks.

How the benchmark works

The benchmark uses eve-bench and the DeepSWE-lean dataset. DeepSWE-lean has 12 feature and bug-fix tasks from DeepSWE v1.1.

All harnesses use the same model and the same tasks.
Each harness does each task 3 times.
Each attempt runs in a new Vercel Sandbox.
Each attempt has a limit of 20 minutes.
The DeepSWE verifier examines the files that the agent leaves.
An attempt passes only when the verifier passes.
A crash or a timeout is a failure.

The line on each correctness row shows the 95% interval. When the lines of two harnesses overlap, the benchmark does not show a difference between them.

How the results update

A maintainer runs the eve-code workflow on main with the deepswe-lean dataset.
The workflow runs eve-code, opencode, and pi. It saves each complete result in the eve-bench result store.
The workflow writes a summary of the result to apps/docs/lib/evals/eve-code-benchmark.json. Then it commits the file to main.
The next docs deployment shows the new results.

The workflow does not publish incomplete runs, single-task runs, or pull request runs. Results can be older than main, because a maintainer starts each run. Select a harness name to open the GitHub Actions run for that result.

Mount the extension

Create an extension file in your agent. The file name sets the mount name, which prefixes every contributed tool and subagent ID (for example, code__grep):

agent/extensions/code.ts

import code from "eve/extensions/code";

export default code({});

code({}) mounts the extension without GitHub or Vercel authentication. Add the options below when the agent needs them.

Configure the extension

Option	Required	Effect
`github.connector`	With `github`	Connector that mints a GitHub token
`github.org`	With `github`	Organization whose repositories the `gh` tool can access
`github.broker`	With `github`	`broker(sandbox, rules)` callback that installs, then removes, the GitHub credential rules
`vercel.connector`	With `vercel`	Connector that requests an app-subject Vercel token
`vercel.delivery`	No	`"firewall"` (default) or `"command"`
`broker`	No	`broker(sandbox, rules)` callback that merges Vercel credential rules into your network policy
`worker.model`	With `worker`	Model for the `worker` subagent. Defaults to `openai/gpt-5.6-terra-fast`
`worker.reasoning`	With `worker`	Reasoning level for the `worker` subagent. Defaults to `xhigh`
`worker.openaiReasoningEffort`	No	Passed to OpenAI models as `reasoningEffort`

agent/extensions/code.ts

import code from "eve/extensions/code";

export default code({
  vercel: { connector: "vercel/acme-bot" },
  worker: { model: "openai/gpt-5.6-terra-fast", reasoning: "high" },
});

Credentials stay outside the sandbox

With firewall delivery, Vercel tokens never enter sandbox processes. Firewall delivery without a top-level broker requires a sandbox provider that exposes setNetworkPolicy(), and fails otherwise. Set vercel.delivery: "command" for providers without a mutable network policy.

The gh tool requests a token scoped to exactly one repository in github.org for each invocation. The sandbox process receives a placeholder GH_TOKEN, and the firewall exchanges it on matching GitHub requests. Your github.broker callback receives the header-transform rules for the command, then null to remove the lease. The callback must preserve your other network rules. GitHub authentication has no command-delivery option.

To use a personal access token or another credential provider, omit the connector and call authenticateGitHub or authenticateVercel from eve/extensions/code/sandbox in your own sandbox lifecycle. These helpers authenticate commands you run yourself; they do not configure the gh tool.

Prepare the sandbox

Install the CLI tooling in your sandbox environment's prepare callback:

agent/sandbox.ts

import { defineSandbox } from "eve/sandbox";
import { VercelSandbox } from "eve/sandbox/vercel";
import { installCodeTooling } from "eve/extensions/code/sandbox";

export const environment = VercelSandbox.environment({
  prepare: async (sandbox) => {
    await installCodeTooling(sandbox, { vercel: true });
  },
});

export default defineSandbox(() => environment.open());

Preparation installs gh, wrappers for gh, vc, and gh-signed-commit, and TypeScript diagnostics. For repositories that require verified signatures, stage your changes and commit with gh-signed-commit.

eve derives the prepared environment generation from the sandbox file and environment options, not from imported helpers. Upgrading eve alone does not rebuild an existing prepared environment.