# Phase 2 Baseline — pre-implementation measurements

**Date:** 2026-05-12
**Targets probed:** lmcp v0.5.4 on `boltzmann.fritz.box:8080/mcp`; OpenAI-compat broker on `hossenfelder.fritz.box:8082`.

This is the Phase 7 (verify) anchor — captures what the world looked like just *before* Phase 2 implementation lands, so post-implementation behavior can be compared against it. Companion to PHASE2.md (manifest).

---

## 1. MCP RPC round-trip timings (cold path, single warm-up)

| RPC | Latency |
|---|---|
| `initialize` | 19 ms |
| `notifications/initialized` (HTTP 202, no body) | 11 ms |
| `tools/list` | 17 ms |
| `tools/call` `list_dir({path:"/tmp"})` (success, ~1 KB result) | 72 ms |
| `tools/call` `read_file({path:"/nonexistent/..."})` (handler-caught failure) | 12 ms |
| `tools/call` `nope_tool` (JSON-RPC -32601 unknown tool) | 12 ms |

LAN-local; sub-100ms for everything but a file-listing payload. Phase 2's
sequential tool-call dispatch won't be the bottleneck — the LLM is.

---

## 2. Fixtures (saved to `/tmp/aish-baseline/`)

| File | Shape |
|---|---|
| `01_initialize.json` | `{result:{protocolVersion, serverInfo:{name,version}, capabilities:{tools:{listChanged:false}}}}` |
| `02_notif_init.body` | empty (HTTP 202) |
| `03_tools_list.json` | `{result:{tools:[{name, description, inputSchema}...]}}` — 7 tools on boltzmann |
| `04_tools_call_ok.json` | `{result:{isError:false, content:[{type:"text", text:"<listing>"}]}}` |
| `05_tools_call_iserror.json` | **see §3 finding** |
| `06_tools_call_unknown.json` | `{error:{code:-32601, message:"Tool not found: nope_tool"}}` |

### Initialize response (compact)

```json
{"id":1,"jsonrpc":"2.0","result":{
    "serverInfo":{"version":"0.1.0","name":"boltzmann-tools"},
    "protocolVersion":"2025-03-26",
    "capabilities":{"tools":{"listChanged":false}}}}
```

### Unknown-tool error (transport-level failure)

```json
{"id":5,"jsonrpc":"2.0","error":{
    "message":"Tool not found: nope_tool","code":-32601}}
```

---

## 3. Baseline finding: `isError` is not a complete failure signal

`read_file({path:"/nonexistent/baseline-probe"})` returned:

```json
{"id":4,"jsonrpc":"2.0","result":{
    "isError":false,
    "content":[{"type":"text","text":"Error: could not read /nonexistent/baseline-probe"}]}}
```

`isError: false` despite an obvious failure. The handler caught the error and put it in `content` text but didn't set the flag.

**Implication for Phase 2 design:** aish cannot rely solely on `result.isError` to decide success/failure of a tool call. The model must read the text content. This actually simplifies Phase 2: just feed `content` straight back as the `role:"tool"` turn body regardless of `isError`. The flag is advisory; the model is the discriminator. (No PHASE2.md amendment needed — §4's "pass-through to the model" stance already accommodates this.)

This is a per-tool boltzmann-lmcp implementation quirk, not a spec issue. Other lmcp deployments may set `isError: true` correctly; aish should still pass content through and not crash on either shape.

---

## 4. Streaming `tool_calls` delta shape (verified against hossenfelder)

For `stream: true` requests with `tools` declared, observed deltas:

```
data: {"choices":[{"delta":{"role":"assistant","content":null}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"...","type":"function",
                                            "function":{"name":"get_weather","arguments":""}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{"}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\""}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"city"}}]}}]}
...
data: {"choices":[{"finish_reason":"tool_calls","delta":{}}]}
data: [DONE]
```

Accumulator rules confirmed:
1. On the first delta containing `tool_calls[i]`: capture `id`, `type`, `function.name`. `arguments` may be empty `""`.
2. On subsequent deltas matching same `index`: concatenate `function.arguments` into the running buffer.
3. `finish_reason: "tool_calls"` closes the set; arguments buffer is parsed as JSON at that point.

Matches PHASE2.md §5 design.

---

## 5. Baseline aish behavior (pre-MCP, what Phase 1 does today)

Sent to hossenfelder with the standard system prompt and **no `tools` field**:

```
user: List the files in /tmp
```

Response (qwen2.5-coder-1.5b via hossenfelder, sans tools):

```
```cmd
dir /tmp
```
```

`finish_reason: stop`, `tool_calls: null`, 9 completion tokens.

The loaded model emits Windows shell syntax in a markdown code-fence, ignoring the system prompt's `CMD:` extraction contract. **No tool_calls path is exercised today** because no tools are declared. This is the empirical "before" of Phase 2 — once MCP servers are wired and a real tool exists (`list_dir({path:"/tmp"})`), the model has a structured path that doesn't depend on getting `CMD:` formatting right.

---

## 6. Known blockers carried into Phase 7 (verify)

Both live in the **boltzmann proxy** (`hossenfelder.fritz.box:8082`), not in aish:

| # | Bug | Affects | Tracking |
|---|---|---|---|
| 1 | SSE buffering — proxy sets `Content-Length` on `text/event-stream` and flushes the whole response at once | streaming visibility (Phase 1) AND streaming tool_calls deltas (Phase 2) | [aish#15](https://git.reauktion.de/marfrit/aish/issues/15) + [[reference-hossenfelder-sse-buffering]] |
| 2 | `model` field routing — every request returns chunks tagged `qwen2.5-coder-1.5b-q4_k_m.gguf` regardless of requested `model`, suggesting the proxy ignores the field | Phase 2 testing against mistral-nemo specifically (the strict-chat-template canary for Q18); also any `:model deep` / `:model cloud` switch | side-finding in #15 triage; needs its own issue when Phase 7 hits it |

Phase 2 implement/verify will proceed against whatever model is loaded.
Full template-strictness verification of Q18 (`role:"tool"` acceptance on
mistral-nemo) waits for bug #2 to be fixed in the boltzmann proxy code.

---

## 7. Module pre-state (Phase 1 head: `5878f73`)

| Module | LOC (incl. comments) | State |
|---|---|---|
| `broker.lua` | 92 | chat + chat_stream, no `tools` field |
| `context.lua` | (per Phase 1) | `pending_exec_output` buffer; no `role:"tool"`; no `tool_calls` on assistant turns |
| `executor.lua` | (per Phase 1) | PTY-backed, `CMD:` extract, no tool dispatch |
| `repl.lua` | 287 | meta cmds, ask_ai stream loop, no `:mcp …`, no tool-call sub-loop |
| `renderer.lua` | 79 | exec frame, streaming text; no tool-call frame |
| `safety.lua` | (per PHASE0 §4) | stub — only the file exists |
| `mcp.lua` | — | does not exist yet |
| `config.lua` | (per user's edits) | models registry; no `mcp = { servers = {...} }` section |

After Phase 2 lands, `git diff main..post-phase-2 --stat` should show:
new `mcp.lua` (substantial), modest growth in `broker.lua` / `context.lua` /
`repl.lua` / `renderer.lua`, finally non-stub `safety.lua`.

---

*End of Phase 2 Baseline — aish*