buzz/crates/buzz-agent/README.md

# buzz-agent

> Minimal, unbreakable ACP-compliant LLM agent. Stdio in, tool calls out. Non-streaming. No persistence. No cleverness.

[ACP](https://agentclientprotocol.com) is the Agent Client Protocol — JSON-RPC 2.0 over stdio between a client (Zed, JetBrains, buzz-acp, …) and an agent. [MCP](https://modelcontextprotocol.io) is how the agent talks to its tools.

`buzz-agent` is the agent.

## What It Is

```
        +--------+   stdio (JSON-RPC 2.0)   +---------------+
        | client | <----------------------> |  buzz-agent |
        +--------+        ACP frames        +---------------+
                                              │            │
                                              │            │ rmcp (stdio)
                                              │            ▼
                                              │       MCP servers
                                              │       (your tools)
                                              ▼
                                            HTTPS
                                              │
                                              ▼
                                  Anthropic Messages API,
                                   OpenRouter, or any OpenAI-compat
                                  (vLLM, llama.cpp, Databricks,
                                   Block Gateway, Ollama, …)
```

A client sends `session/prompt`. The agent loops: call the LLM → get tool calls → run them via MCP → feed results back → repeat. The loop terminates when the LLM stops asking for tools, the round cap is hit, or the client cancels.

The agent's **output is its tool calls**. Generated text is forwarded to the client as `agent_message_chunk` updates, but the real work happens in the tools. The LLM call is non-streaming — one HTTP POST, one response.

## Quick Start

```bash
# Build
cargo build --release -p buzz-agent

# Run against Anthropic
BUZZ_AGENT_PROVIDER=anthropic \
ANTHROPIC_API_KEY=sk-ant-... \
ANTHROPIC_MODEL=claude-sonnet-4-5 \
  ./target/release/buzz-agent

# Or any OpenAI-compatible endpoint
BUZZ_AGENT_PROVIDER=openai \
OPENAI_COMPAT_API_KEY=sk-... \
OPENAI_COMPAT_MODEL=gpt-5 \
OPENAI_COMPAT_BASE_URL=https://api.openai.com/v1 \
  ./target/release/buzz-agent

# Or OpenRouter
BUZZ_AGENT_PROVIDER=openrouter \
OPENROUTER_API_KEY=sk-or-v1-... \
OPENROUTER_MODEL=anthropic/claude-sonnet-4.5 \
  ./target/release/buzz-agent

# Or Databricks model serving via OAuth 2.0 PKCE
BUZZ_AGENT_PROVIDER=databricks \
DATABRICKS_HOST=https://dbc-...cloud.databricks.com \
DATABRICKS_MODEL=goose-claude-4-6-sonnet \
  ./target/release/buzz-agent
```

That's the whole setup. The agent reads JSON-RPC frames from stdin, writes them to stdout, and logs to stderr.

## ACP Transcript

A complete round-trip. Lines starting with `→` are client→agent (stdin); `←` are agent→client (stdout). Each line is one newline-terminated JSON value. Comments are not part of the wire.

```jsonc
// 1. Handshake.
→ {"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":1,"clientCapabilities":{}}}
← {"jsonrpc":"2.0","id":1,"result":{
    "protocolVersion":1,
    "agentCapabilities":{
      "loadSession":false,
      "promptCapabilities":{"image":false,"audio":false,"embeddedContext":false},
      "mcpCapabilities":{"http":false,"sse":false}
    },
    "agentInfo":{"name":"buzz-agent","version":"0.1.0"}
  }}

// 2. Open a session. The client passes the MCP servers to spawn.
→ {"jsonrpc":"2.0","id":2,"method":"session/new","params":{
    "cwd":"/tmp",
    "mcpServers":[{"name":"echo","command":"/usr/local/bin/echo-mcp","args":[],"env":[]}]
  }}
← {"jsonrpc":"2.0","id":2,"result":{"sessionId":"ses_a1b2c3d4e5f6a7b8"}}

// 3. Prompt. The agent loops until the LLM stops calling tools.
→ {"jsonrpc":"2.0","id":3,"method":"session/prompt","params":{
    "sessionId":"ses_a1b2c3d4e5f6a7b8",
    "prompt":[{"type":"text","text":"echo hello"}]
  }}

// 4. Agent emits tool_call (status: pending) — visible to the UI.
← {"jsonrpc":"2.0","method":"session/update","params":{
    "sessionId":"ses_a1b2c3d4e5f6a7b8",
    "update":{
      "sessionUpdate":"tool_call",
      "toolCallId":"toolu_01XYZ",
      "title":"echo__say",
      "kind":"other",
      "status":"pending",
      "rawInput":{"text":"hello"}
    }
  }}

// 5. Agent moves the call to in_progress, runs the MCP tool, then completed.
← {"jsonrpc":"2.0","method":"session/update","params":{
    "sessionId":"ses_a1b2c3d4e5f6a7b8",
    "update":{"sessionUpdate":"tool_call_update","toolCallId":"toolu_01XYZ","status":"in_progress"}
  }}
← {"jsonrpc":"2.0","method":"session/update","params":{
    "sessionId":"ses_a1b2c3d4e5f6a7b8",
    "update":{
      "sessionUpdate":"tool_call_update",
      "toolCallId":"toolu_01XYZ",
      "status":"completed",
      "content":[{"type":"content","content":{"type":"text","text":"hello"}}]
    }
  }}

// 8. The model sees the result, decides it's done, and the prompt resolves.
← {"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}
```

That's ACP. Three request methods (`initialize`, `session/new`, `session/prompt`), one inbound notification (`session/cancel`), and three outbound update variants (`agent_message_chunk`, `tool_call`, `tool_call_update`). The full server is hand-rolled in `main.rs`.

## Configuration

Everything is environment variables. No flags, no config files. (We are a subprocess; subprocess config is environment.)

| Variable | Default | Notes |
|---|---|---|
| `BUZZ_AGENT_PROVIDER` | — | Required. `anthropic`, `openai`, `openrouter`, `databricks`, or `databricks_v2`. No implicit fallback — the agent errors at startup when this is unset. |
| `ANTHROPIC_API_KEY` | — | Required when provider=anthropic. |
| `ANTHROPIC_MODEL` | — | Required when provider=anthropic. |
| `ANTHROPIC_BASE_URL` | `https://api.anthropic.com` | |
| `ANTHROPIC_API_VERSION` | `2023-06-01` | |
| `OPENAI_COMPAT_API_KEY` | — | Required when provider=openai. |
| `OPENAI_COMPAT_MODEL` | — | Required when provider=openai. |
| `OPENAI_COMPAT_BASE_URL` | `https://api.openai.com/v1` | Point at vLLM, llama.cpp, Ollama, etc. |
| `OPENAI_COMPAT_API` | `auto` | `auto` \| `chat` \| `responses`. `auto` picks Responses for `*.openai.com`, Chat Completions everywhere else. |
| `OPENROUTER_API_KEY` | — | Required when provider=openrouter. |
| `OPENROUTER_MODEL` | — | Required when provider=openrouter. Use OpenRouter's `vendor/model` id, e.g. `anthropic/claude-sonnet-4.5`. |
| `OPENROUTER_BASE_URL` | `https://openrouter.ai/api/v1` | |
| `DATABRICKS_HOST` | — | Required when provider=databricks or provider=databricks_v2. |
| `DATABRICKS_MODEL` | — | Required when provider=databricks or provider=databricks_v2. |
| `DATABRICKS_TOKEN` | — | Optional static bearer escape hatch. If unset, Databricks uses browser OAuth + refresh cache. |
| `BUZZ_AGENT_SYSTEM_PROMPT` | built-in | Inline system prompt. |
| `BUZZ_AGENT_SYSTEM_PROMPT_FILE` | — | File path. Mutually exclusive with the above. |
| `BUZZ_AGENT_MAX_ROUNDS` | `0` | Tool-loop iteration cap. 0 = unlimited. |
| `BUZZ_AGENT_MAX_OUTPUT_TOKENS` | `32768` | Per LLM call. Headroom for large tool-call inputs (e.g. file writes via heredoc); Sonnet 4 / Opus 4 cap at 64K. |
| `BUZZ_AGENT_MAX_CONTEXT_TOKENS` | `200000` | Provider context window used by the handoff gate. |
| `BUZZ_AGENT_MAX_HANDOFFS` | `10` | Max context handoffs per session before falling back to truncation. |
| `BUZZ_AGENT_LLM_TIMEOUT_SECS` | `240` | Max seconds with no response bytes before abandoning an LLM call (per-read inactivity, not wall-clock). |
| `BUZZ_AGENT_TOOL_TIMEOUT_SECS` | `660` | Per-tool call timeout in seconds |
| `BUZZ_AGENT_MAX_PARALLEL_TOOLS` | `8` | Max concurrent tool calls per turn (1 = sequential) |
| `BUZZ_AGENT_MAX_SESSIONS` | unlimited | Max concurrent ACP sessions. Sessions are cheap; default has no cap. |
| `BUZZ_AGENT_MAX_LINE_BYTES` | `4194304` | 4 MiB. Hard cap on inbound JSON-RPC frames. |
| `BUZZ_AGENT_MAX_HISTORY_BYTES` | `1048576` | 1 MiB. Old turns are evicted past this. |
| `BUZZ_AGENT_MAX_TOOL_RESULT_TEXT_BYTES` | `51200` | 50 KiB. Per-result cap on tool-output text; oversize is middle-elided (head + tail kept) with an inline marker. Images are exempt. |
| `BUZZ_AGENT_REQUIRE_REPLY` | `0` (`1` on mesh) | `1` enables the [reply guard](#reply-guard) — remind the model to publish when a turn is about to end with nothing posted to Buzz. Desktop defaults it to `1` for Buzz shared-compute agents. |


## Reply Guard

Off by default, except on Buzz shared-compute (mesh) agents, where Buzz Desktop
sets `BUZZ_AGENT_REQUIRE_REPLY=1` automatically. With it enabled, a turn that is
about to end without any recognized attempt to post to Buzz gets a reminder that
its assistant text is invisible to humans, and is rerolled.

This exists because a Buzz agent's reasoning and tool output are not shown to
anyone. A turn that does real work and never posts is a silent failure — the
requester waits on a result that was produced and thrown away.

Mesh agents get it by default because they run on small local models, which are
the ones most likely to do the work and then end the turn without publishing it.
Setting `BUZZ_AGENT_REQUIRE_REPLY=0` on the agent, persona, or global env opts a
mesh agent back out; the default never overrides an explicit value.

**Advisory, never a trap.** At most two reminders, then the turn ends whether or
not anything was published. The guard catches accidental omission; it does not
compel speech. The reminder text explicitly licenses silence, because the
built-in system prompt says publishing is optional and silence is often the
correct outcome.

**Recognition contract.** A turn counts as having replied when it issues a call
that:

- resolves to a registered, non-hook tool (a hallucinated tool name is rejected
  at preflight and never runs, so it must not disarm the guard),
- whose qualified name ends in `__shell` — i.e. the bare tool name is exactly
  `shell`, which is `buzz-dev-mcp`'s shell tool and any other server's, and
- whose `command` argument contains `messages send` or `reactions add`.

`messages send` also covers `messages send-diff`. Reactions count because the
built-in prompt directs agents to react rather than post a bare
acknowledgement, so nagging an agent that reacted would punish documented
behavior.

Detection is checked **after** the per-turn tool-call cap
(`MAX_TOOL_CALLS_PER_TURN`) is applied: a publish-shaped call that was discarded
never ran.

**It recognizes an attempt, not a successful publish.** Only the command text is
inspected, never the exit status. A send that fails still satisfies the guard —
which is fine, since a failed send already returns a non-zero exit and error
JSON to the model, louder feedback than a reminder.

**Known limits**, both deliberate. A command assembled at runtime (`$CMD`) or
buried in a wrapper script is missed, so that turn is reminded despite having
posted. Text that merely quotes a send (`echo "buzz messages send"`) matches, so
that turn is not reminded. Missing a real post is the expensive direction, and
substring matching is the forgiving one there. Neither edge is pinned by a test;
the matcher is free to improve.

**Budget.** Reminders ride the existing `_Stop` gate and share
`BUZZ_AGENT_STOP_MAX_REJECTIONS` — the outer cap on every end-turn objection.
At the default 3 both reminders fit; at 1 only one does; at 0 the guard is off
along with the hooks. A round carrying both a `_Stop` hook objection and a
reminder costs one rejection and delivers both texts. This is not a new
lifecycle hook — see [MCP_DRIVEN_HOOKS.md](../../docs/MCP_DRIVEN_HOOKS.md).


## Providers

`buzz-agent` speaks a few HTTP dialects. Pick with `BUZZ_AGENT_PROVIDER`.

| Provider | `BUZZ_AGENT_PROVIDER` | Endpoint (auto) | Tested with |
|---|---|---|---|
| Anthropic | `anthropic` | `POST {base}/v1/messages` | claude-sonnet-4-5, claude-opus-4 |
| OpenAI | `openai` | `POST {base}/responses` | gpt-5, gpt-5-mini, o4-mini, gpt-4o |
| vLLM | `openai` | `POST {base}/chat/completions` | any tool-calling model |
| llama.cpp | `openai` | `POST {base}/chat/completions` | any tool-calling GGUF |
| Ollama | `openai` | `POST {base}/chat/completions` | llama3.1, qwen2.5-coder |
| Block Gateway | `openai` | `POST {base}/chat/completions` | gpt-5, claude |
| OpenRouter | `openrouter` | `POST {base}/chat/completions` | anything they route (extended-thinking replay, provider-agnostic tool calling) |
| Databricks | `databricks` | `POST {host}/serving-endpoints/{model}/invocations` | goose-claude-4-6-sonnet |
| Databricks AI Gateway v2 | `databricks_v2` | `POST {host}/ai-gateway/{provider}/v1/...` | databricks-gpt-5-5, databricks-claude-opus-4-7 |

If `BUZZ_AGENT_PROVIDER=anthropic` is selected without `ANTHROPIC_API_KEY`, `BUZZ_AGENT_PROVIDER=openai` is selected without `OPENAI_COMPAT_API_KEY`, or `BUZZ_AGENT_PROVIDER=openrouter` is selected without `OPENROUTER_API_KEY`, the agent returns an error — there is no implicit fallback to another provider.

`provider=openai` speaks two HTTP dialects: the [Responses API](https://platform.openai.com/docs/api-reference/responses) (`/v1/responses`, required for GPT-5 / o-series tool-calling on OpenAI's own service) and the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat) (`/chat/completions`, the broadly-supported OpenAI-compatible wire format).

By default (`OPENAI_COMPAT_API=auto`) the agent picks **Responses** when `OPENAI_COMPAT_BASE_URL` points at an `*.openai.com` host and **Chat Completions** everywhere else. Pin the choice explicitly with `OPENAI_COMPAT_API=chat` or `OPENAI_COMPAT_API=responses` for providers that diverge from the default (e.g. a Responses-compatible self-hosted gateway).

`provider=openrouter` is first-class, not routed through `provider=openai`: it speaks OpenAI's Chat Completions wire format but with OpenRouter-specific extensions layered on top —

- `reasoning.effort` is set on the request when reasoning effort is configured. The request deliberately carries no `provider.require_parameters` filter: that filter routes only to endpoints advertising every parameter in the body, and 83 of 274 tools-capable OpenRouter models do not advertise `reasoning`, so it turns an effort setting into a hard 404 on a valid model id. A model that cannot reason answers without reasoning instead.
- The response's `reasoning_details` array (opaque extended-thinking payload) is captured and replayed byte-for-byte on the next turn's assistant message, so multi-turn tool use keeps the model's chain-of-thought.
- `anthropic/*` models get Anthropic-style `cache_control` breakpoints injected on the system message and the last two user messages.
- Retryable statuses (429 and typed `provider_overloaded` 503) honor the documented `Retry-After` header (clamped to a small ceiling — see `RETRY_AFTER_CAP_SECS` in `llm.rs` — since the sleep happens outside `BUZZ_AGENT_LLM_TIMEOUT_SECS`); 502 and untyped 503 retry with jittered backoff instead. `401` is treated as an expired/invalid key and refreshed once, while `402` (no credits) and `403` (guardrail/moderation/permission) fail immediately without retry.

`Provider` is a Rust `enum` with one `match` in `Llm::complete`. There is no trait, no `Box<dyn>`, no async-trait. Adding a provider is a `match` arm and one `body`/`parse` pair in `llm.rs`.

## MCP Servers

The client passes MCP server specs in `session/new`. The agent spawns each one as a stdio subprocess, calls `tools/list`, and merges everything into a single tool catalog the LLM sees. Tool names are namespaced as `server__tool` (double underscore separator). Bare tool names containing `__` are rejected at registration.

Example: a single echo MCP server.

```json
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "session/new",
  "params": {
    "cwd": "/work",
    "mcpServers": [
      {
        "name": "echo",
        "command": "/usr/local/bin/echo-mcp",
        "args": ["--mode", "stdio"],
        "env": [
          { "name": "ECHO_VERBOSE", "value": "1" }
        ]
      }
    ]
  }
}
```

Multiple servers: just add more entries. Tool calls fan out to the right server by namespace prefix.

**Transport: stdio only.** No HTTP, no SSE. We advertise this in `agentCapabilities` (`mcpCapabilities.http: false`, `mcpCapabilities.sse: false`); spec-compliant clients won't ask for what we don't have.

## Security Model

The trust boundary is **the operator who launched the agent**. The harness, MCP server binaries, and API keys are all trusted. Untrusted input — model output, tool results, prompts — is bounded.

| Boundary | Mechanism |
|---|---|
| Stdout discipline | Single-consumer `mpsc` channel feeding stdout. No two tasks can interleave bytes. All logs go to stderr. |
| MCP child env | Whitelist (`PATH`, `HOME`, `TERM`, `LANG`, `LC_ALL`, `TMPDIR`) plus what the client explicitly passes. Your `ANTHROPIC_API_KEY` does not leak into MCP children. |
| MCP child lifetime | Process group via `setpgid(0,0)` in `pre_exec`. On transport break or shutdown: `killpg(SIGKILL)`. Grandchildren die too. |
| Server poisoning | After a timeout or transport break, the offending server is marked dead. Future calls trigger a lazy restart with exponential backoff. Other servers keep working. |
| Frame size | `BUZZ_AGENT_MAX_LINE_BYTES` (default 4 MiB). Oversize → connection killed. |
| LLM response size | 16 MiB hard cap. Both `Content-Length` precheck and streaming-buffer cap. |
| Cancellation | `tokio::select! { biased; _ = cancel.changed() => ... }` at every loop boundary. Cancel always wins the race. |
| Session isolation | Unlimited concurrent sessions by default (configurable via `BUZZ_AGENT_MAX_SESSIONS`). One prompt per session at a time. Each session gets its own MCP servers. |
| `tool_use ↔ tool_result` pairing | Encoded in the type system. Every `ToolCall` and `ToolResult` carries a `provider_id: String` (not `Option`). |

### Bounded Everything

| Limit | Default | Where |
|---|---|---|
| Inbound JSON-RPC frame | 4 MiB | `BUZZ_AGENT_MAX_LINE_BYTES` |
| Single prompt | 1 MiB | `MAX_PROMPT_BYTES` |
| History window | 1 MiB | `BUZZ_AGENT_MAX_HISTORY_BYTES` |
| LLM response body | 16 MiB | `MAX_LLM_RESPONSE_BYTES` |
| LLM error body | 4 KiB | `MAX_LLM_ERROR_BODY_BYTES` |
| Tool result body (total, incl. images) | 8 MiB | `MAX_TOOL_RESULT_BYTES` |
| Tool result text | 50 KiB | `BUZZ_AGENT_MAX_TOOL_RESULT_TEXT_BYTES` |
| MCP servers / session | 16 | `MAX_MCP_SERVERS` |
| Tools / session | 128 | `MAX_TOOLS_PER_SESSION` |
| Tool description bytes | 1 KiB | `MAX_DESCRIPTION_BYTES` |
| Tool schema bytes | 4 KiB | `MAX_SCHEMA_BYTES` (oversize → replaced with `{}`) |
| Tool calls per turn | 64 | `MAX_TOOL_CALLS_PER_TURN` |
| Loop rounds | 0 (unlimited) | `BUZZ_AGENT_MAX_ROUNDS` |
| LLM read inactivity timeout | 240 s | `BUZZ_AGENT_LLM_TIMEOUT_SECS` |
| Tool call timeout | 660 s | `BUZZ_AGENT_TOOL_TIMEOUT_SECS` |

## What This Is NOT

A short list, because the answer is mostly "no":

- **Not a framework.** No plugins, no recipes, no slash commands, no modes. MCP servers can participate in agent lifecycle via [hook tools](../../docs/MCP_DRIVEN_HOOKS.md) (`_Stop`, `_PostCompact`), but these are advisory, fail-open, and budget-bounded — not a plugin system.
- **Not streaming.** One non-streaming HTTP POST per round. The LLM's generated text is forwarded to the client as `agent_message_chunk`, but there is no token-level streaming.
- **Not persistent.** Everything is in-memory, per-process. No SQLite. When context fills, the agent summarizes its own history and continues (context handoff). No external persistence.
- **Not an SDK.** This is a binary. The protocol seam is stdin/stdout. Use it from any language.
- **Not a UI.** No TUI, no web, no notifications. The client renders.
- **Not authenticated.** API keys come from env. Use systemd, Docker secrets, or a wrapper.
- **Not networked MCP.** Stdio transport only. No HTTP/SSE MCP transport.
- **Not load-able.** No `session/load`. We advertise `loadSession: false`.
- **Not a router.** No agent-to-agent, no fan-out, no orchestration. One model. One loop.

**Concurrency model:**

```
                  ┌──── reader task ──────────┐
                  │  (stdin → JSON-RPC → ...) │
                  │                           │
   stdin ─────────┤   dispatch                │
                  │     │                     │
                  │     ├── initialize        │  (sync reply)
                  │     ├── session/new       │  (sync reply)
                  │     ├── session/prompt ───┼─── spawn ──> prompt task
                  │     │                     │              │
                  │     ├── session/cancel ───┼─> watch::send│ (biased select wins)
                  │     │                     │              │
                  └───────────────────────────┘              │
                                                             │
                  ┌── writer task ────────────────┐          │
   stdout ────────┤  mpsc<WireMsg> consumer       │<─────────┘
                  │  (the only stdout writer)     │
                  └───────────────────────────────┘
```

One reader, one writer, up to 8 concurrent prompt tasks (one per session).

## Building

```bash
cargo build --release -p buzz-agent
```

## Testing

```bash
cargo test -p buzz-agent
```

Test strategy is **real subprocess, no mocks**:

- **Fake LLM** — `tests/fake_llm.rs` and the helpers in `tests/regressions.rs` spin up a real `tokio::net::TcpListener` on port 0, parse `Content-Length`, and return scripted JSON. No HTTP mocking library.
- **Fake MCP server** — `tests/bin/fake_mcp.rs` is a separate binary controlled by env vars: `FAKE_MCP_HANG_INIT`, `FAKE_MCP_TOOL_DELAY`, `FAKE_MCP_SPAWN_GRANDCHILD`, etc. Each fault path is a real process being abused.
- **Regression tests are the changelog.** Each `#[test]` in `regressions.rs` is named for the bug it locks down: `assistant_text_preserved_across_prompts`, `cancel_leaves_history_valid_for_next_prompt`, `mcp_init_timeout_kills_child`, `oversize_line_kills_connection`. Read them in order to learn the protocol's failure modes.