marfrit/aish - aish - marfrit's space

marfrit/aish

Fork 0

Commit Graph

Author	SHA1	Message	Date
marfrit	ffead3986c	docs/PHASE4: review fold-in — flock for race, Norris suppression, summarizer self-amp Independent review found 1 BLOCKER + 3 CONCERNs + 4 NITs. R-B1 (BLOCKER): TOCTOU race on memory.jsonl — two aish processes scanning the same file compute identical next_ids. Resolution: flock(LOCK_EX \| LOCK_NB) on the fd in M.open_memory, held until close. Bundled into commit #1 (per reviewer: cannot defer because adding flock retroactively means reopening the handle). Requires ffi/libc.lua extension: flock cdef + LOCK_EX/LOCK_NB/LOCK_UN constants + M.flock wrapper. R-C1 (CONCERN, closes Q33): [background] block suppressed when ctx.norris_active. Avoids ~16K of redundant tokens per 8-step Norris run. Norris already anchors via its goal in the NORRIS suffix; memory items rarely change step-to-step planning. R-C2 (CONCERN): summarizer self-amplification — running :memory summarize twice in one session would feed the prior summarize call's assistant turn into the next input. Resolution: operate on the session log file (history.load(session_path)) instead of ctx:to_messages(), and tag prior summarize turns with meta="summarize" so they're filterable. R-C3 (CONCERN, cosmetic): §5 diagram clarified that DEFAULT_SYSTEM_PROMPT already carries the Phase 2 MCP block statically — not a separate dynamic block in v1. NITs N1-N4 folded inline: N1 forget no-op for unknown id surfaces a status N2 path note: memory.jsonl is sibling of sessions/, no collision N3 item-id invariants: id >= 1; meta header has no id; tombstones with non-matching targets are no-ops N4 :memory inject semantics explicit (replace ctx.memory_items from a fresh load + LRU-by-ts truncation) §3 module-changes table grew a new ffi/libc.lua row. §12 commit #1 description tightened — flock work bundled inline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-13 04:50:43 +00:00
marfrit	2146b909f8	docs/PHASE4: analyze — surface confirmed, counter strategy locked A1. history.lua surface lines up cleanly for the memory additions — no structural refactor; pure additive functions mirroring the session pattern. A2. Counter persistence: scan at open, cache next_id in handle. O(n) load (n bounded by curation, ~hundreds), no sidecar file. Persisted ids let forget-tombstones target items even across restarts. A3. System-prompt suffix order locked: DEFAULT (carrying Phase 2 MCP block baked in) → Phase 4 [background] → Phase 3 NORRIS. Token cost measured: default ~174 toks, +NORRIS ~364 toks, +NORRIS+2KB background ~865 toks. Well within typical context budgets. No manifest amendments needed — §3/§5 already match. Findings recorded inline as Phase 7 anchors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-13 04:47:01 +00:00
marfrit	bea717534c	docs/PHASE4: formulate — memory.jsonl + startup injection + :memory meta Phase 4 formulate manifest. Three pillars per PHASE0 §11 row 4: memory.jsonl persistent cross-session store, startup context injection into the system prompt, and the :memory management surface + opt-in :memory summarize for candidate extraction. Resolutions baked in via §2: - Storage: append-only JSONL at <history.dir>/memory.jsonl - Format: {id, ts, kind, content, tags?, source?} - Kinds: fact / pref / context (lightly typed v1) - Forget: tombstone append, resolve at load (set-based) - Cadence: manual :memory summarize only in v1; auto-trigger Q-listed - Inject: dynamic [background] block on system prompt, capped at 2000 chars by default; LRU-by-ts selection if over-budget - Order: DEFAULT → MCP block → [background] → NORRIS suffix (Norris last so it dominates when active) New module surfaces: history.lua M.open_memory / memory:add / memory:forget / M.load_memory context.lua ctx.memory_items + [background] composer repl.lua :remember, :memory add/list/forget/clear/inject/summarize config.lua commented-out memory = {...} example Open questions (Q31-Q36) tracked in §11: Q31 auto-summarize trigger (manual v1; auto-on-quit candidate) Q32 in-place edit vs forget+re-add Q33 Norris-mode interaction (proposal: both blocks stay) Q34 split prefs into a dedicated prompt section? Q35 redaction of sensitive content during summarize Q36 duplicate detection on :memory add 5-commit roadmap in §12 (history → context → repl → summarize → config). No new module files. No substrate amendments to PHASE0 — entirely additive on top of Phase 1's history.lua pattern and Phase 3's dynamic-suffix pattern in context.lua. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-13 04:25:57 +00:00

Author

SHA1

Message

Date

marfrit

ffead3986c

docs/PHASE4: review fold-in — flock for race, Norris suppression, summarizer self-amp

Independent review found 1 BLOCKER + 3 CONCERNs + 4 NITs.

R-B1 (BLOCKER): TOCTOU race on memory.jsonl — two aish processes
  scanning the same file compute identical next_ids. Resolution:
  flock(LOCK_EX | LOCK_NB) on the fd in M.open_memory, held until
  close. Bundled into commit #1 (per reviewer: cannot defer because
  adding flock retroactively means reopening the handle). Requires
  ffi/libc.lua extension: flock cdef + LOCK_EX/LOCK_NB/LOCK_UN
  constants + M.flock wrapper.

R-C1 (CONCERN, closes Q33): [background] block suppressed when
  ctx.norris_active. Avoids ~16K of redundant tokens per 8-step
  Norris run. Norris already anchors via its goal in the NORRIS
  suffix; memory items rarely change step-to-step planning.

R-C2 (CONCERN): summarizer self-amplification — running :memory
  summarize twice in one session would feed the prior summarize
  call's assistant turn into the next input. Resolution: operate
  on the session log file (history.load(session_path)) instead
  of ctx:to_messages(), and tag prior summarize turns with
  meta="summarize" so they're filterable.

R-C3 (CONCERN, cosmetic): §5 diagram clarified that
  DEFAULT_SYSTEM_PROMPT already carries the Phase 2 MCP block
  statically — not a separate dynamic block in v1.

NITs N1-N4 folded inline:
  N1 forget no-op for unknown id surfaces a status
  N2 path note: memory.jsonl is sibling of sessions/, no collision
  N3 item-id invariants: id >= 1; meta header has no id; tombstones
     with non-matching targets are no-ops
  N4 :memory inject semantics explicit (replace ctx.memory_items
     from a fresh load + LRU-by-ts truncation)

§3 module-changes table grew a new ffi/libc.lua row.
§12 commit #1 description tightened — flock work bundled inline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-13 04:50:43 +00:00

marfrit

2146b909f8

docs/PHASE4: analyze — surface confirmed, counter strategy locked

A1. history.lua surface lines up cleanly for the memory additions —
    no structural refactor; pure additive functions mirroring the
    session pattern.

A2. Counter persistence: scan at open, cache next_id in handle.
    O(n) load (n bounded by curation, ~hundreds), no sidecar file.
    Persisted ids let forget-tombstones target items even across
    restarts.

A3. System-prompt suffix order locked: DEFAULT (carrying Phase 2 MCP
    block baked in) → Phase 4 [background] → Phase 3 NORRIS. Token
    cost measured: default ~174 toks, +NORRIS ~364 toks, +NORRIS+2KB
    background ~865 toks. Well within typical context budgets.

No manifest amendments needed — §3/§5 already match. Findings recorded
inline as Phase 7 anchors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-13 04:47:01 +00:00

marfrit

bea717534c

docs/PHASE4: formulate — memory.jsonl + startup injection + :memory meta

Phase 4 formulate manifest. Three pillars per PHASE0 §11 row 4:
memory.jsonl persistent cross-session store, startup context injection
into the system prompt, and the :memory management surface +
opt-in :memory summarize for candidate extraction.

Resolutions baked in via §2:
  - Storage:  append-only JSONL at <history.dir>/memory.jsonl
  - Format:   {id, ts, kind, content, tags?, source?}
  - Kinds:    fact / pref / context (lightly typed v1)
  - Forget:   tombstone append, resolve at load (set-based)
  - Cadence:  manual :memory summarize only in v1; auto-trigger Q-listed
  - Inject:   dynamic [background] block on system prompt, capped at
              2000 chars by default; LRU-by-ts selection if over-budget
  - Order:    DEFAULT → MCP block → [background] → NORRIS suffix
              (Norris last so it dominates when active)

New module surfaces:
  history.lua  M.open_memory / memory:add / memory:forget / M.load_memory
  context.lua  ctx.memory_items + [background] composer
  repl.lua     :remember, :memory add/list/forget/clear/inject/summarize
  config.lua   commented-out memory = {...} example

Open questions (Q31-Q36) tracked in §11:
  Q31 auto-summarize trigger (manual v1; auto-on-quit candidate)
  Q32 in-place edit vs forget+re-add
  Q33 Norris-mode interaction (proposal: both blocks stay)
  Q34 split prefs into a dedicated prompt section?
  Q35 redaction of sensitive content during summarize
  Q36 duplicate detection on :memory add

5-commit roadmap in §12 (history → context → repl → summarize → config).
No new module files. No substrate amendments to PHASE0 — entirely
additive on top of Phase 1's history.lua pattern and Phase 3's
dynamic-suffix pattern in context.lua.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-13 04:25:57 +00:00

3 Commits