Phase 2 commit #5 per docs/PHASE2.md §12. Streaming broker grows
tool-call support without taking a dependency on mcp.lua (caller
supplies the tools array — B5 from review).
chat_stream signature widens to (cfg, msgs, on_delta, opts):
opts.tools - optional array, passed to the request body as the
OpenAI-shape tools field. OMITTED entirely when nil or
empty (#tools == 0) — some servers reject "tools": [].
on_delta callback shape widens to (kind, payload):
kind = "text", payload = string (Phase 1 path; unchanged
semantics, signature
changes from (delta) to
("text", delta))
kind = "tool_call", payload = {id, name, arguments}
emitted ONCE per call on
finish_reason "tool_calls"
after the streaming
accumulator pulls
fragmented JSON-string
arguments together.
Accumulator behavior:
- Keyed by delta.tool_calls[i].index.
- If index is absent on a delta (some llama.cpp builds omit it on
single-call streams; C2 in review), default to 0 with a one-shot
stderr debug status per stream.
- id and name captured from the opening delta of each slot.
- function.arguments concatenated across all deltas as the raw
JSON-string; caller (repl.lua / future Phase 2 commit #6) does
dkjson.decode.
- On finish_reason "tool_calls" the accumulator emits all collected
calls in index order and resets.
M.chat external contract unchanged (C1): wrapper now uses the new
(kind, payload) shape internally but exposes the same text-string
return. No caller of M.chat passes opts.tools so tool_call kinds are
silently dropped.
repl.lua minimal companion edit: ask_ai's chat_stream callback updated
to the new shape. Text path unchanged; tool_call kinds are no-op
placeholders until commit #6 lands the sub-loop. Keeps Phase 1 streaming
functional between #5 and #6.
Smoke-tested against hossenfelder/8082 (post-#23 fix):
- text-only: ok=true, kind="text" deltas received
- with opts.tools: model emitted one tool_call,
accumulator collected id + name=get_weather + args={"city":"Paris"}
correctly across fragmented deltas
- opts.tools={}: server accepted (field omitted as required)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 streaming consumer per PHASE1.md §3.
broker.chat_stream(model_cfg, messages, on_delta) -> true | (nil, err)
broker.chat(model_cfg, messages) -> content | (nil, err)
(now a thin buffer over
chat_stream)
The HTTP shape unifies on stream:true. on_event from ffi/curl.post_sse
decodes each event's JSON, extracts choices[1].delta.content, and calls
on_delta(content) for non-empty string deltas. The `[DONE]` sentinel is
filtered. SSE-framed error envelopes ({"error":{"message":...}} arriving
as data:) surface as "api: ..." errors.
build_request is factored out so chat_stream and (future) any
non-streaming consumer share URL/body/header construction.
Live verification against hossenfelder fast preset:
- chat_stream("Count one to five..."): 9 incremental deltas streamed
token-by-token, assembled to "1 2 3 4 5"
- chat("Reply with exactly: pong"): "pong" returned via buffer
Error envelope path is correct by inspection but not exercised live —
hossenfelder passes through bogus model names rather than rejecting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0 implementation per PHASE0.md §6.
M.chat(model_cfg, messages) -> content_string | (nil, errmsg)
Builds the OpenAI-compat JSON body:
{ model, messages, stream: false, temperature: model_cfg.temperature ?? 0.2 }
Sends Content-Type and (optionally) Authorization Bearer pulled from
model_cfg.key_env's process environment. Default timeout 60s; overridable
per-model via model_cfg.timeout_ms.
Error surfaces split:
"transport: ..." curl-side (TCP/TLS/timeout)
"decode: ..." non-JSON response body
"api: ..." OpenAI-style { error: { message } } envelope
"broker.chat: no choices[1].message.content..." shape miss
Tested against four canned mock responses (nc -lN listener feeding
HTTP/1.0 + Connection: close so EOF terminates the body): happy path,
api error envelope, raw-text non-JSON, empty choices[]. The on-wire
request body verified as well: POST path, headers, model/messages/
temperature/stream JSON.
Live test against a real llama.cpp/hossenfelder endpoint deferred per
issue #12 (broker endpoint configuration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- README, .gitignore, CLAUDE.md (project conventions)
- docs/PHASE0.md — full Phase 0 manifest (locked substrate)
- 10 root .lua modules + 4 ffi/ bindings, all stubs raising NotImplemented
with module-scoped responsibilities matching the manifest
- config.lua wired to current dirac/hossenfelder endpoints (qwen-coder-7b
snappy/32k + cloud via OpenRouter through hossenfelder)
File names match docs/PHASE0.md §4 exactly. Module bodies fill in across
later phases; the tree shape is locked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
marfrit
claude-noether