lurker: say in the ROOM when a model swap happened, not only in the journal (#118, lurker half)

The lurker falls back from claude to opencode on any dead end — right, since posting a
failure verbatim helps nobody. But the reply then carried only the SECOND model's name,
so a thin answer read as "the configured model had little to say" when the configured
model had in fact never finished.

Live on 2026-08-02: a Fable architecture run for @reviewer died on error_max_turns, the
lurker fell back to opencode/deepseek, deepseek burned its own budget on the same search
and answered nothing useful. That Fable had run at all appeared solely in
`journalctl --user -u bullpen-lurker@reviewer` — exactly where nobody looks while reading
a disappointing reply. Same defect idea #118 fixed for the grinder: report the model that
actually served, and say so when it was not the one asked for.

The banner only appears on an actual swap; the common case stays as quiet as before.

tests/test_lurker_fallback_visible.py pins all three: the banner names both models on a
swap, no banner when the configured model answers, and a double dead-end still returns a
real diagnosis rather than a banner wrapped around nothing. Mutation-checked — cutting the
announcement fails exactly the first test and leaves the other two green. (The first
attempt at that mutation silently did not apply, which made three green tests meaningless;
redone via a file with an explicit marker assertion.)
This commit is contained in:
2026-08-02 12:08:43 +02:00
parent 8b09e6dd0d
commit d2eb9820f2
2 changed files with 85 additions and 1 deletions
+8 -1
View File
@@ -325,7 +325,14 @@ def run_agent(prompt):
# If opencode ALSO dead-ends, hand back the more informative of the two so the room at
# least sees a real diagnosis instead of two layers of "(no result)".
if oc and not oc.lstrip().startswith(f"(@{NICK}"):
return oc
# SAY IT IN THE ROOM, not only on stderr. The reply carries the second model's
# name, so a thin answer reads as "the configured model had little to say" when
# what actually happened is that the configured model never finished. On
# 2026-08-02 a Fable architecture run died on max_turns, deepseek answered
# instead, and the only trace was journalctl. Same defect bullpen idea #118 fixed
# for the grinder: report what SERVED you, and say so when it was not what you asked.
return (f"⚠ answered by the fallback: {MODEL or 'claude'} {why}, "
f"served by opencode {_ocmodel()}\n\n{oc}")
return ans if (ans and not ok) else oc
return ans
+77
View File
@@ -0,0 +1,77 @@
"""A model swap must be visible in the ROOM, not only in the journal (#118, lurker half).
The lurker falls back from claude to opencode on any dead end — a good behaviour, since
posting a failure verbatim helps nobody. But the reply then carried only the SECOND
model's name, so a thin answer read as "the configured model had little to say" when what
really happened was that the configured model never finished.
Live on 2026-08-02: a Fable architecture run for @reviewer died on `error_max_turns`, the
lurker silently fell back to opencode/deepseek, and deepseek burned its own budget on the
same search and answered nothing. The only record that Fable had run at all was
`journalctl --user -u bullpen-lurker@reviewer` — exactly where nobody looks when reading a
disappointing reply. Same defect bullpen idea #118 fixed for the grinder: report the model
that actually served you, and say so when it was not the one you asked for.
"""
import importlib.machinery
import importlib.util
import os
import sys
import pytest
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@pytest.fixture
def lurk(monkeypatch, tmp_path):
monkeypatch.setenv("HOME", str(tmp_path))
(tmp_path / "spec_lurker").mkdir()
sys.modules.pop("lurk", None)
argv, sys.argv = sys.argv, ["bullpen-lurker", "spec"]
try:
ld = importlib.machinery.SourceFileLoader(
"lurk", os.path.join(REPO, "lurker", "bullpen-lurker"))
mod = importlib.util.module_from_spec(importlib.util.spec_from_loader("lurk", ld))
ld.exec_module(mod)
finally:
sys.argv = argv
return mod
def test_fallback_announces_itself_in_the_reply(lurk, monkeypatch):
monkeypatch.setattr(lurk, "RUNTIME", "claude", raising=False)
monkeypatch.setattr(lurk, "MODEL", "fable", raising=False)
monkeypatch.setattr(lurk, "run_claude", lambda p: ("(no result, rc 1)", False))
monkeypatch.setattr(lurk, "run_opencode", lambda p: "here is the architecture verdict")
monkeypatch.setattr(lurk, "_ocmodel", lambda: "bosch/deepseek")
out = lurk.run_agent("some prompt")
assert "architecture verdict" in out, "the fallback's answer must still come through"
assert "fable" in out, "the reply must name the model that was ASKED for and failed"
assert "deepseek" in out, "…and the model that actually served"
def test_no_banner_when_the_configured_model_answers(lurk, monkeypatch):
"""The common case must not get noisier — same posture as the grinder's model tag."""
monkeypatch.setattr(lurk, "RUNTIME", "claude", raising=False)
monkeypatch.setattr(lurk, "MODEL", "fable", raising=False)
monkeypatch.setattr(lurk, "run_claude", lambda p: ("a fine answer", True))
monkeypatch.setattr(lurk, "run_opencode", lambda p: pytest.fail("must not fall back"))
out = lurk.run_agent("some prompt")
assert out == "a fine answer"
assert "" not in out
def test_double_dead_end_still_returns_a_diagnosis(lurk, monkeypatch):
"""When BOTH runtimes fail there is nothing to announce — the room needs the error,
not a banner wrapped around an empty string."""
monkeypatch.setattr(lurk, "RUNTIME", "claude", raising=False)
monkeypatch.setattr(lurk, "MODEL", "fable", raising=False)
monkeypatch.setattr(lurk, "NICK", "spec", raising=False)
monkeypatch.setattr(lurk, "run_claude", lambda p: ("(@spec: claude API error)", False))
monkeypatch.setattr(lurk, "run_opencode", lambda p: "(@spec: opencode gave nothing)")
monkeypatch.setattr(lurk, "_ocmodel", lambda: "bosch/deepseek")
out = lurk.run_agent("some prompt")
assert "spec" in out and "error" in out.lower()