Phase 4 commit #1 per docs/PHASE4.md §12. Two file changes bundled
because R-B1 (flock for race-free single-writer enforcement) cannot
be deferred — adding it retroactively means reopening the memory
handle.
ffi/libc.lua extensions:
- cdef flock(int fd, int op), open(...), lseek(int, long, int)
- constants LOCK_EX=2, LOCK_NB=4, LOCK_UN=8
- M.flock(fd, op) wrapper returning (true) on success or
(false, errmsg) — errmsg is the strerror text so callers can
surface "Resource temporarily unavailable" cleanly to the user.
history.lua additions (Phase 4 section appended at end):
- M.open_memory(path) -> handle | nil, err
Opens the file via libc.open(2) (need integer fd for flock —
io.open's FILE* doesn't expose it), takes flock(LOCK_EX | LOCK_NB).
Returns "memory.jsonl held by another aish process" on lock-held.
Scans existing content for max id; caches as handle.next_id.
Writes meta header on first creation (no id, ignored at load).
- handle:add(kind, content, tags?, source?) -> id
Assigns next id; appends one JSONL item with auto-timestamp.
kind ∈ {fact, pref, context} enforced via assert.
- handle:forget(target_id)
Appends a tombstone {id, ts, kind:"forget", target}.
- handle:close()
Releases fd (flock auto-released on close).
- M.load_memory(path) -> items_table
Reads all lines, builds forget-target set from kind=="forget"
entries, returns active items as an array sorted by ts desc.
Items without id (meta header) silently dropped. Tombstones with
non-matching targets are no-ops (N3 invariant).
Round-trip test passes:
- open empty file → next_id=1
- add 3 items → ids 1, 2, 3
- forget id 2 (appends tombstone)
- reopen → next_id correctly advances past the tombstone (=5)
- load_memory → 2 active items (id 1 + id 3); tombstone resolved
- lock-held detection: second open while first held → fails with
"memory.jsonl held by another aish process" message
- close releases the lock; reopen after release succeeds
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3 commit #5 per docs/PHASE3.md §12. Wires safety.norris_step
(commit #4) into the REPL with the user-facing surface.
ffi/readline.lua extensions (A1 + R-C4):
- rl_insert_text + rl_redisplay added to ffi.cdef block; M.insert_text
and M.redisplay wrappers exposed.
- M.bind: removed `:free()` on previous callback. Now keeps every
bound callback pinned for process lifetime in `_pinned` list
(alongside `_bound[seq]` for current lookup). Avoids the
use-after-free window between unbind and rebind that R-C4 flagged.
Memory cost is bounded — one closure per key sequence binding.
context.lua Norris suffix (R-C3 / §8):
- to_messages() composes a dynamic NORRIS MODE block onto the
system prompt when ctx.norris_active is set. The block carries
ctx.norris_goal so eviction of the user's "[norris] goal:" turn
doesn't lose the anchor. Returns to plain system prompt when
Norris exits.
repl.lua Norris driver:
- prompt() now shows ⚡ marker when ctx.norris_active per PHASE0.md §9.
- \C-n bound to a real handler — inserts ":norris " at the cursor
(replaces Phase 1 status placeholder).
- run_norris(goal) function: sets norris_active + norris_goal,
appends a "[norris] <goal>" user turn, renders the banner, then
loops calling safety.norris_step with an injected helpers table
until a terminal status returns. Renders the closing banner.
- norris_halt(): the [N] proceed/skip/abort prompt called by
safety.norris_step via helpers.halt. Empty input → abort (safe).
- dispatch_tool(): factored from the Phase 2 ask_ai code so
safety.norris_step can call it.
- norris_exec(): factored exec path for autonomous mode (skips
the interactive run_shell cd-status renderer).
- :norris <goal> meta — launches autonomous mode
- :norris off meta — drops Norris flag (rare; usually 'abort')
- :safety patterns meta — lists active is_destructive rules
- :safety check <cmd> meta — probes a hypothetical command
End-to-end mock-driven test:
Submitted ":norris find files in /tmp" → banner → step 1 emits
tool_call (auto_approved per policy) → dispatched → frame rendered
→ step 2 emits "GOAL: complete" → sub-loop exits → DONE banner.
2 broker invocations, no stalls.
config.lua safety example block lands in commit #6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 7 verify finding from TC #26 against :model cloud:
HTTP 400 from openrouter→Amazon Bedrock:
"tools.0.custom.name: String should match pattern
'^[a-zA-Z0-9_-]{1,128}$'"
Anthropic via Bedrock validates tool names against that regex and
rejects dots. PHASE2 originally chose "." as the namespace separator
("boltzmann.list_dir"); OpenAI tolerated it, Bedrock does not.
Separator switched to "__" (two underscores) everywhere — internal
API matches on-wire shape, no transformation layer:
- repl.lua:
- tools_schema builds "alias__name"
- dispatch_tool_call splits via "^(.-)__(.+)$" (non-greedy → leftmost __)
- :mcp tool parser uses same split
- :mcp tools formatter prints "alias__name"
- HELP block shows <alias__name>
- safety.lua confirm_tool_call: alias.* glob → alias__* glob
- config.lua example block: keys rewritten
- docs/PHASE2.md: amendment header added; §1, §2 row, §3 config.lua
row, §5 wire-shape JSON examples, §6 auto_approve schema, §7
meta-cmd table, §12 plan all updated. Original "." references
preserved in commit history.
Constraint: aliases must not themselves contain "__" so the parse
stays unambiguous. Tool names from MCP servers may have underscores
freely.
Second fix bundled — uninformative broker error:
Previously "broker error: transport: HTTP response code said error"
Now "broker error: transport: HTTP 400: {full body snippet}"
ffi/curl.lua M.post_sse changes:
- FAILONERROR no longer set (was hiding the response body).
- raw_body accumulator added alongside the SSE buffer; captures
every byte regardless of SSE shape.
- After perform, check status_code via curl_easy_getinfo. On >=400,
return (nil, "HTTP <code>: <body[:400]>"). 2xx unchanged.
- End-of-stream SSE flush only runs on 2xx (no false event on
error bodies that aren't SSE-shaped).
- Phase 1 callers reading just first return slot stay correct.
End-to-end verified:
- :model cloud + tools=[boltzmann__read_file ...] +
"Use boltzmann__read_file with path=/etc/hostname" →
Claude emits tool_call with name="boltzmann__read_file",
args='{"path": "/etc/hostname"}'. ok=true, transport clean.
- Force-bad tool name "bad.name.with.dots" → err string carries
the full bedrock 400 with the regex-pattern message visible.
TC #26 (sub-loop end-to-end) is now testable against cloud — the
error that blocked it is resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First commit of Phase 2 per docs/PHASE2.md §12. Three changes bundled:
mcp.lua (new, 153 lines):
- M.connect(url, opts) returns a Session.
- Session:initialize() round-trips initialize + notifications/initialized
+ tools/list. Caches tools for session lifetime (lmcp announces
capabilities.tools.listChanged = false; no refetch).
- Session:list_tools() returns the cached tool list.
- Session:call_tool(name, args) returns (result_table, kind) where
kind ∈ {"ok", "handler_error", "rpc_error", "transport_error"} per
the §4 error split. Folded HTTP-level failure into transport_error.
- Per-server Bearer auth via opts.auth_token or opts.auth_env env-var
indirection.
- Captures protocolVersion mismatch as a warning string rather than
aborting (lmcp doesn't negotiate — N3 in review).
ffi/curl.lua extension:
- Add curl_easy_getinfo to ffi.cdef.
- Pre-cast as getinfo_long; helper get_response_code() fetches
CURLINFO_RESPONSE_CODE (decimal 2097154 = CURLINFOTYPE_LONG | 2).
- M.post now returns (body, status_code) on transport success;
(nil, errmsg) on libcurl failure stays unchanged. Phase 1 callers
reading only the first slot are unaffected.
docs/PHASE0.md §4:
- Insert `mcp.lua` between broker.lua and router.lua per PHASE2.md §9.
- Module-stability invariant clarified: rename prohibition is what
matters; adding new files is additive.
Smoke-test passes for all four kinds against boltzmann lmcp v0.5.4:
- initialize: ok (7 tools cached)
- list_dir /tmp: ok (1.2KB content)
- read_file /nonexistent: ok (boltzmann's baseline §3 quirk —
isError:false even on failure; content is authoritative)
- nope_tool: rpc_error (code=-32601)
- wrong auth: transport_error (HTTP 401)
- unreachable host: transport_error (DNS failure)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CONCERNs from the Phase 1 review pass:
ffi/curl.lua:
- SSE write_cb body is now pcall-wrapped. A Lua error in on_event (or
in the parse loop itself) is captured into cb_error and surfaced
after curl_easy_perform rather than propagating across the FFI
callback boundary (which LuaJIT documents as process-fatal). The
EOS flush path gets the same shield. Errors return
(nil, "callback: <msg>") from post_sse.
history.lua:
- sh_singlequote() escapes shell metacharacters; the mkdir -p and
ls -1 shell-outs no longer double-quote (where $(...) and $VAR
still expand) — single-quote with embedded-' escaping is the
safe form.
- M.load now returns (turns, meta) instead of (meta, turns). turns
is ALWAYS a table on success, never nil-when-no-header; failure
path is the unambiguous (nil, err). Callers can `if not turns
then` without the previous ambiguity. repl.lua :resume updated
to the new shape.
repl.lua :resume:
- Refuse to resume into a non-empty ctx — silent overwrite was the
Q15 default, but the review surfaced the no-undo / no-warning
failure mode. User must :reset (or :save then re-launch) to
express intent. The current session's on-disk log is unaffected
either way.
NITs:
- ffi/libc.lua READ_BUF: comment noting it's module-shared and
Phase 1 has no reentrant readers; revisit when that changes.
- PHASE1.md §7: \C-x\C-c reservation pinned to Phase 3 ("deferred
from Phase 1 — no consumer here") rather than the previous
dangling "(or here)".
Regression suite verifies:
- history.load new signature on success + failure paths
- shell-quoted history.dir with $ doesn't trip
- aish scripted run: ctx with 2 turns refuses :resume anchor with
a clear status; user must :reset first
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 review caught a structural gap: executor.exec only drained the
PTY master fd, never forwarded user keystrokes — vim/less/htop/nano
would render and hang on input. PHASE1.md §5 specified bidirectional
multiplex but only the read leg landed. tcgetattr/tcsetattr were also
missing, so even with input forwarding the parent's line discipline
would buffer until newline (breaking single-key UIs).
ffi/libc:
- struct termios opaque buffer + tcgetattr/tcsetattr + cfmakeraw
- M.set_raw(fd) saves termios + applies cfmakeraw; returns saved or
(nil, err) when fd isn't a tty (scripted / piped-stdin runs)
- M.restore_termios(fd, saved)
- struct pollfd + M.poll (POLLIN constant)
executor:
- multiplex(sess): poll(stdin, master); reads master on any revents
(POLLHUP fires when child closes its slave end, not POLLIN — the
revents != 0 check catches both); forwards stdin keystrokes to
master; loop exits when master read returns 0 (EOF / child gone)
- stdin polling is only enabled when stdin_is_tty (set_raw succeeded);
piped-stdin runs (tests / scripted) would otherwise drain queued
aish commands into the child of the *current* cmd, swallowing them
- raw mode is restored before returning so the user lands back at the
aish prompt in canonical mode
renderer + repl:
- exec_output(out, code) split into exec_begin() (top rule, before
spawn) + exec_end(code) (closing rule with exit, after wait). PTY
multiplex streams the body live to stdout in between; the renderer
never re-prints the body.
PHASE1.md §3:
- tcgetattr/tcsetattr changed from "optional" to "required for
single-key UIs to work — done-criteria #2"; poll added to the libc
row description.
Verified:
- non-interactive smoke (echo / false / exit 7 / ls /nonexistent /
printf multi-line) — all exit codes correct, output streamed live,
a\nb\nc\n preserved byte-for-byte
- scripted-stdin run reaches all expected lines (no stdin draining
into a non-interactive child)
- aish prompt + framed exec block + exit-code line all render in
correct order
Live interactive verification (vim / less / htop in a real terminal)
still needs a user-test pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 readline binding wiring per PHASE1.md §7.
ffi/readline:
M.bind(seq, lua_fn) -> bool
Wraps lua_fn as a C callback (signature `int (int, int)` per
readline's rl_command_func_t) and registers it via
rl_bind_keyseq(seq, cb). Returns true on success (rl returns 0).
Trampolines are pinned in module-local state so they outlive the
bind call — readline retains the function pointer for the process
lifetime. Rebinding the same seq frees the previous trampoline.
Bound handlers are pcall-wrapped so a Lua error doesn't crash
readline's input loop.
repl:
Binds \C-n to a no-op that emits
"[aish] Norris mode not yet implemented (Phase 3)"
Verifies the mechanism end-to-end; Phase 3 (Norris autonomous mode)
replaces the body with the actual toggle.
Smoke covers bind / rebind-same-seq (exercises the :free path) /
bind-different-seq with no errors. Live keyboard verification waits
on user-test.
Phase 1's 8(+1) inner loop is now functionally through `implement`;
next inner phase is `verify` (review pass) followed by memory-update.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 streaming substrate per PHASE1.md §4.
curl.post_sse(url, body, headers, on_event, timeout_ms)
-> true | (nil, errmsg)
Reuses the Phase 0 WRITEFUNCTION hook. Each chunk delivery accumulates
into a per-request buffer; the buffer is drained for complete events
(\n\n-terminated). Each event's `data: ...` field(s) are joined per the
SSE spec and passed to on_event(data_string) synchronously. `:` comment
lines (keepalives) are filtered.
The `[DONE]` sentinel is passed through to on_event as-is (broker.lua
filters it — this module stays HTTP-layer only, no JSON / OpenAI shape
knowledge).
Two robustness items:
- End-of-stream flush: the final event may lack \n\n if the server
closes-on-EOF immediately after the last data: line (some llama.cpp
builds, plain HTTP/1.0 close-on-EOF feeds). Post-perform, any
remaining buffer is parsed as one last event.
- FAILONERROR: a non-2xx response surfaces as a CURLcode error rather
than silently feeding the error body into the SSE parser.
Smoke:
[1] canned events via nc listener: 3 events parsed in order
[2] chunk-split mid-event ("Hel" + sleep + "lo..."): correctly
reassembled across two WRITEFUNCTION deliveries
[3] LIVE against hossenfelder.fritz.box:8082 fast preset with
stream:true: response "pong" assembled from incremental deltas;
4 raw events (role + 1 content + finish_reason + [DONE])
Next: broker.lua chat_stream that decodes the OpenAI delta shape on
top of this and exposes on_delta(content_string) for renderer streaming.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 PTY substrate per PHASE1.md §5. Replaces Phase 0's io.popen
sentinel-echo path with a real PTY so interactive cmds (vim, less,
htop) work and exit-status comes from waitpid instead of parsing a
sentinel out of stdout.
API:
pty.spawn(cmd) -> session | (nil, err)
session:read(count) -> (data, n) ; n == 0 means EOF
session:write(data) -> bytes
session:close() ; closes master_fd; child gets SIGHUP
session:wait(options) -> (kind, val) ; "exit"/"signal"/"other"/nil
session:signal(sig) -> ok ; kill(pid, sig)
Child branch execs `/bin/sh -c cmd`, preserving Phase 0's shell-
interpretation semantics (quoting, redirection, pipes still work).
The PTY makes vim/less/htop functional because the child gets a real
tty for line discipline instead of a pipe.
Loader uses the versioned-soname fallback idiom (util / util.so.1 /
util.so.0) so a runtime-only host without libutil-dev works.
Smoke covers: echo hello (exit 0), false (1), exit 7, bogus binary
(sh's 127), multi-line printf, cat bidirectional (write ping -> read
echo+cat output -> close master -> child exits via SIGHUP).
Next: executor.lua swap from popen+sentinel to pty.spawn. That commit
also retires the §7 amendment paragraph (no longer needed once popen
is gone).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends Phase 0's chdir/errno/strerror with the syscalls that ffi/pty
needs to drive a forkpty'd child: waitpid (with WIFEXITED / WEXITSTATUS
/ WIFSIGNALED / WTERMSIG decoders), read, write, close, kill.
Status-word macros are reproduced from glibc bits/waitstatus.h using
the LuaJIT `bit` library. M.waitpid returns a structured (kind, value)
rather than the raw status word — callers don't have to know the
encoding:
"exit", N — normal exit, N is exit code
"signal", N — killed by signal N
"other", raw — stopped/continued (Phase 1 doesn't trace those)
nil, err — syscall failure
M.read / M.write / M.close / M.kill mirror their syscall return shape
with errno-string surfacing on failure. Read uses a shared 4 KiB
buffer for the common case; larger reads allocate a fresh buffer.
Smoke covers the chdir regression (still works), all four status
decoders against known status words, pipe round-trip for read/write/
close, EOF -> ("", 0), invalid-fd close -> false, kill(self, 0)
success, kill(bogus, 0) failure.
waitpid is not exercised by the smoke (needs a real child); that
arrives with ffi/pty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0 binding per PHASE0.md §6. M.post(url, body, headers, timeout_ms)
uses CURLOPT_{URL, POST, POSTFIELDS, HTTPHEADER, WRITEFUNCTION, NOSIGNAL,
TIMEOUT_MS, USERAGENT} on a fresh easy handle, capturing the response
into a Lua string via a closure-based WRITEFUNCTION callback.
curl_easy_setopt is variadic; LuaJIT's variadic FFI dispatch needs
ffi.new() per argument otherwise. Pre-cast to three concrete signatures
(long / void* / const char*) bypasses that — cleaner and matches the
lua-curl idiom.
Robust loader: tries `curl`, `curl.so.4`, `curl-gnutls.so.4` so a
runtime-only host (no libcurl-dev installed) just works. Same idiom
as ffi/readline.
Smoke against a local nc listener: request was correctly framed
(POST path, Content-Type + X-Test headers, Content-Length matches
JSON body length) and the canned response was captured into the
returned Lua string.
SSE streaming for Phase 1 reuses this same WRITEFUNCTION hook —
chunks arrive incrementally, the closure consumes them as they come.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0 binding per PHASE0.md §9. M.readline(prompt) returns the line
as a Lua string (the C buffer is freed via libc free immediately after
ffi.string copies it) or nil on EOF. M.add_history skips empty lines.
Loader handles the case where libreadline-dev's unversioned
`libreadline.so` symlink isn't installed — falls through to
`readline.so.8` (current Debian/Arch ALARM) and `.so.7` (older)
before giving up. This trips on noether-the-LXD: only the runtime
package is present.
Smoke (stdin from heredoc, two lines + EOF):
p1> hello world -> "hello world"
p2> second line -> "second line"
p3> -> nil (EOF)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smallest Phase 0 module per CLAUDE.md §4 implementation order.
M.chdir(path) returns (true) or (false, errmsg) — errmsg via
strerror(__errno_location()[0]). Glibc errno is thread-local
behind __errno_location() rather than a plain global, hence the
indirect access.
Verified against PHASE0.md §7 expectation: a libc.chdir() persists
across subsequent io.popen() calls (popen's child inherits the
parent's wd), which is the property executor.lua relies on for `cd`
interception. Smoke:
libc.chdir("/tmp"); io.popen("pwd"):read("*l") --> /tmp
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- README, .gitignore, CLAUDE.md (project conventions)
- docs/PHASE0.md — full Phase 0 manifest (locked substrate)
- 10 root .lua modules + 4 ffi/ bindings, all stubs raising NotImplemented
with module-scoped responsibilities matching the manifest
- config.lua wired to current dirac/hossenfelder endpoints (qwen-coder-7b
snappy/32k + cloud via OpenRouter through hossenfelder)
File names match docs/PHASE0.md §4 exactly. Module bodies fill in across
later phases; the tree shape is locked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
marfrit
claude-noether