broker: opts.tools passthrough + streaming tool_call accumulator

Phase 2 commit #5 per docs/PHASE2.md §12. Streaming broker grows
tool-call support without taking a dependency on mcp.lua (caller
supplies the tools array — B5 from review).

chat_stream signature widens to (cfg, msgs, on_delta, opts):
  opts.tools  - optional array, passed to the request body as the
                OpenAI-shape tools field. OMITTED entirely when nil or
                empty (#tools == 0) — some servers reject "tools": [].

on_delta callback shape widens to (kind, payload):
  kind = "text",      payload = string         (Phase 1 path; unchanged
                                                semantics, signature
                                                changes from (delta) to
                                                ("text", delta))
  kind = "tool_call", payload = {id, name, arguments}
                                                emitted ONCE per call on
                                                finish_reason "tool_calls"
                                                after the streaming
                                                accumulator pulls
                                                fragmented JSON-string
                                                arguments together.

Accumulator behavior:
  - Keyed by delta.tool_calls[i].index.
  - If index is absent on a delta (some llama.cpp builds omit it on
    single-call streams; C2 in review), default to 0 with a one-shot
    stderr debug status per stream.
  - id and name captured from the opening delta of each slot.
  - function.arguments concatenated across all deltas as the raw
    JSON-string; caller (repl.lua / future Phase 2 commit #6) does
    dkjson.decode.
  - On finish_reason "tool_calls" the accumulator emits all collected
    calls in index order and resets.

M.chat external contract unchanged (C1): wrapper now uses the new
(kind, payload) shape internally but exposes the same text-string
return. No caller of M.chat passes opts.tools so tool_call kinds are
silently dropped.

repl.lua minimal companion edit: ask_ai's chat_stream callback updated
to the new shape. Text path unchanged; tool_call kinds are no-op
placeholders until commit #6 lands the sub-loop. Keeps Phase 1 streaming
functional between #5 and #6.

Smoke-tested against hossenfelder/8082 (post-#23 fix):
  - text-only: ok=true, kind="text" deltas received
  - with opts.tools: model emitted one tool_call,
    accumulator collected id + name=get_weather + args={"city":"Paris"}
    correctly across fragmented deltas
  - opts.tools={}: server accepted (field omitted as required)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

This commit is contained in:

Markus Fritsche

2026-05-12 14:20:32 +00:00

parent c736d0e129

commit efdc7281c7

2 changed files with 93 additions and 15 deletions

									
										repl.lua
									
		+9
		-3
	
												View File
												
				@@ -114,9 +114,15 @@ function M.run(config)

				        local parts = {}

				        local ok, err = broker.chat_stream(active_cfg, ctx:to_messages(),

				            function(delta)

				                parts[#parts + 1] = delta

				                renderer.assistant_delta(delta)

				            function(kind, payload)

				                -- Phase 2: callback shape widened to (kind, payload).

				                -- tool_call kinds are handled by the sub-loop landing in

				                -- commit #6; this commit ships only the text path so Phase

				                -- 1 streaming stays functional between #5 and #6.

				                if kind == "text" then

				                    parts[#parts + 1] = payload

				                    renderer.assistant_delta(payload)

				                end

				            end)

				        renderer.assistant_flush()