aish/docs/PHASE9.md

# aish — Phase 9 Manifest

**Project:** aish — AI-augmented conversational shell
**Document:** Phase 9 Requirements, Architecture & Design Decisions
**Status:** Plan (formulate + analyze + baseline complete; tree at `4f5c3ae`)
**Date:** 2026-05-16

**Analyze + baseline findings (2026-05-16) — 5/6 open Qs resolved
in-place; Q-P4 deferred to implement-time verify:**

A1. **main.lua load_config surface clean.** `load_config(opts)` at
    `main.lua:53` returns `(cfg, path)` for the user config. Adding
    a project-overlay wrapper that calls it then walks for `.aish.lua`
    is additive — no refactor of the existing 4-tier resolution.

A2. **No new FFI needed for walk-up.** `io.open(candidate, "rb")` is
    sufficient for existence check; `libc.getcwd()` from Phase 6
    provides the starting point. No new C bindings.

A3. **Q-P2 RESOLVED via probe (B1 below): use `sha256sum`** — GNU
    coreutils ships it everywhere aish targets. Single-shell-out
    pattern; output: `<digest>  <path>` → `cut -d' ' -f1` for the
    hex digest. No new module dependency.

A4. **Q-P1 RESOLVED: trust prompt AFTER `aish: loaded config`
    status.** The user sees what user-config is in play first, then
    decides about the overlay. Natural ordering.

A5. **Q-P3 RESOLVED: don't log walk-up path by default.** Too noisy
    on every startup. If debugging "why isn't my project file
    found?", `:config show` after startup will reveal the walk
    result (declined-or-not-found is visible). Verbose-mode walk
    log is v2 polish.

A6. **Q-P5 RESOLVED: `:config show` shows top-level only by default.**
    Nested tables collapsed to `{key1, key2, ...}` (just the inner
    table's keys for orientation). `:config show full` for the
    deep dump. Keeps the diagnostic surface tractable.

A7. **Q-P6 RESOLVED: project layer CAN set `secrets.vault`** — it's
    part of the trust prompt's scope. User accepting the prompt
    accepts that the project file may redirect secrets. The
    in-memory secrets session is built AFTER config resolution, so
    a project-set `secrets.vault` IS honored.

A8. **rl.readline at startup (Q-P4 — deferred).** Phase 4's
    `:memory summarize` candidate-prompt path also calls
    `rl.readline` early (in metas; not pre-loop). The trust prompt
    fires BEFORE the main loop opens — earlier than any existing
    rl.readline call site. **Implement-time check**: smoke-test
    that rl.readline behaves correctly when called from
    `load_config_with_overlay` before `M.run` ever fires. If it
    misbehaves, fall back to a `printf "..." + read` shell-out for
    the trust prompt.

A9. **Walk-up performance is fine** — at most ~10 levels from a
    typical cwd to $HOME, each `io.open` is ~10us. Total walk
    cost < 1ms even on slow filesystems.

A10. **Trust file race**: two aish instances starting concurrently
     could double-write to `~/.aish/trusted-projects`. JSONL append
     semantics handle this OK (each writes one complete line); a
     duplicate trust entry is harmless. No flock needed (unlike
     memory.jsonl per Phase 4 where the writer SOR was important).

A11. **Sandboxed env for dofile?** Out of scope per §8. The trust
     prompt IS the gate; we accept full Lua execution post-trust.

A12. **Bootstrap chicken-egg**: project's `.aish.lua` could set
     `secrets.vault` which would change WHICH secrets are loaded.
     A12 paths through cleanly: user config loaded → project
     overlay merged → effective config passed to M.run → M.run
     reads `config.secrets.vault` (now possibly the project's) →
     secrets_session built. Order is correct; no chicken-egg.

**Baseline finding:**

B1. `sha256sum` (GNU coreutils 9.7) and `openssl dgst -sha256` agree
    bit-for-bit on the same input file. Both present on noether.
    sha256sum chosen for simpler output parsing (digest in first
    whitespace-separated field; openssl needs `awk '{print $NF}'`).
    Per A3 resolution; documented in Q-P2.

PHASE0 is the locked substrate; PHASE1-8 are layered on top. This manifest
specifies what Phase 9 adds — **project-local config overlay (`.aish.lua`)**:
a per-project config file in or above cwd that merges onto the user's
global config, letting a repo ship its own permission rules, model
presets, skills, hooks, etc. without modifying anyone's `~/.config`.

PHASE0 §11 amendment to add the Phase 9 row lands in the same commit as
this formulate doc.

---

## 1. Scope of Phase 9

Four pillars:

1. **Project-config resolution + walk-up** — at startup, walk up
   from cwd looking for `.aish.lua`. Walk stops at the first found
   file OR at `$HOME` OR at filesystem root (whichever comes first —
   filesystem-root reached without a hit means "no project config").
   The found path is the project layer; absence is a no-op (existing
   resolution path unchanged for users who don't ship project config).

2. **Merge semantics (shallow over user-config)** — load the global
   config first, then `dofile` the project `.aish.lua` and merge its
   top-level keys ONTO the user config. Shallow merge: project's
   `models = {...}` REPLACES the user's entire `models` block (not
   per-model). Predictable; users who want to add ONE model layer
   it deliberately or write a complete `models` block in their
   project file.

3. **Trust prompt + persistent record** — first time aish encounters
   a `.aish.lua` at a given path, prompt the user to trust it
   (`[aish] trust <path>? [y/N]`). On `y`, record the path's
   absolute path AND content hash in `~/.aish/trusted-projects`
   (one JSON line per entry: `{path, sha256, ts}`). On subsequent
   startups: load only if the recorded hash still matches; if the
   file changed since trust, re-prompt. On `n` or empty: skip the
   project layer for this session.

4. **`:config show` meta** — print the resolved config sources
   (which file contributed which top-level key), plus a sanitized
   dump of the effective config (token-bearing fields like
   `auth_token` masked). Useful for debugging when "why doesn't
   my project policy apply?" comes up.

**Phase 9 is done when:**

- A repo with `.aish.lua` in its root opens correctly: aish prompts
  to trust on first encounter, loads + merges on subsequent startups
  (when the hash still matches), and the resulting config behavior
  visibly reflects the project layer (e.g., project-set
  `permissions = { allow = ... }` allow-rules fire).
- `.aish.lua` walk-up finds the file from a nested cwd (e.g.,
  `~/src/aish/docs/` finds `~/src/aish/.aish.lua`).
- Walking past `$HOME` stops (doesn't search `/home/` or `/`).
- Mutating a trusted `.aish.lua` re-prompts (hash mismatch).
- `:config show` lists each source path with the keys it provided.
- Existing configs without any `.aish.lua` behave like Phase 8
  (Phase 8 regression coverage).

---

## 2. Technology Decisions (delta from Phase 8)

| Decision | Choice | Rationale |
|---|---|---|
| Walk-up start | `libc.getcwd()` at startup | Matches existing convention (Phase 6 `:tree` cwd capture). |
| Walk-up stop | `$HOME` OR filesystem root | Don't search outside the user's home — limits attack surface. If no `.aish.lua` between cwd and $HOME, no project layer. |
| Project file name | `.aish.lua` (dotfile) | Matches `.envrc` / `.tool-versions` convention; gitignore-friendly. |
| Merge semantics | Shallow top-level | Predictable; deep merge surprises users when they redefine an array (Lua tables-as-arrays don't merge cleanly). Project users who want to add a single MCP server can copy the user's full `mcp = {...}` block and append. |
| Trust mechanism | Explicit prompt; persist absolute-path + sha256 to `~/.aish/trusted-projects` | Matches `direnv allow` posture. Defense against hostile cloned repos that ship malicious `.aish.lua` (would-be RCE on `cd` + `aish` start). |
| Re-prompt trigger | sha256 mismatch on the recorded path | Trust the BYTES, not just the path — content change = re-prompt. |
| Trust file format | JSONL: `{path, sha256, ts}` per line | Append-only; readable; trivially manageable by hand. |
| Trust file mode | 0600 (matches secrets vault in Phase 5/13) | Local-user trust scope; not a secret per se but defensive. |
| `dofile` execution context | Whatever `dofile` provides (full Lua env) | Project file is arbitrary Lua because that's what the user accepted at trust-prompt. No sandbox; the prompt is the gate. |
| Reload on cd | NO — config resolved at startup only | Mid-session config mutation is a complexity tax. `cd` into a different project means restarting aish. Document. |
| Status line on load | `[aish] project config: <path> (overlaid on <user-config>)` at startup | Visibility — user always knows when project layer is active. |
| `:config show` shape | Lists each source path with the top-level keys it contributed | Diagnoses "why isn't my project rule applying?" cases. Token-bearing fields masked (`auth_token: <set>` rather than the value). |

---

## 3. Module Changes

| File | State after Phase 8 | Phase 9 changes |
|---|---|---|
| `main.lua` | `load_config(opts)` walks $AISH_CONFIG → ~/.config/aish → ./config.lua | Wrap with `load_with_project_overlay(opts)` that finds the user config (existing logic) AND walks up from cwd for `.aish.lua`; if both found, merge project ONTO user and return merged. Records source-per-key for `:config show`. |
| `ffi/libc.lua` | getcwd, chdir, isatty, flock | Add `stat` for filesystem checks during walk-up (or use `io.open(path,"r")` for existence — simpler, no new FFI). |
| `repl.lua` | All the metas including `:config` (nope — no :config yet) | New `:config show` meta. Source-map carried on a module-local set at startup; meta reads it. |
| `history.lua` | session log, memory.jsonl | New helpers: `M.read_trusted(path)` returns set of trusted entries; `M.add_trusted(path, target_path, sha256)` appends. Mode 0600 enforced. |
| `config.lua` (the user's global; not the in-tree example) | n/a | No change. The in-tree `config.lua` becomes a template that project overlays can replace top-level keys of. |
| `docs/PHASE0.md` | §11 lists phases 0-8; §10 resolution order | Amendment: add Phase 9 row to §11; update §10 to mention project overlay. |

No new module files in v1. The hashing logic (sha256) — `openssl dgst -sha256` shelled out (or use `sha256sum`). Both POSIX-portable. Avoid vendoring a Lua sha256 since we already have `openssl` / `sha256sum` available everywhere aish runs.

---

## 4. Pillar 1+2 — Resolution + Merge

### Walk-up

```lua
local function _find_project_config()
    local libc = require("ffi.libc")
    local home = os.getenv("HOME")
    if not home then return nil end
    local dir = libc.getcwd()
    if not dir then return nil end

    -- Don't walk OUTSIDE $HOME. If cwd isn't inside $HOME, no
    -- project search.
    if dir:sub(1, #home) ~= home then return nil end

    while dir and #dir > 0 do
        local candidate = dir .. "/.aish.lua"
        local f = io.open(candidate, "r")
        if f then f:close(); return candidate end
        if dir == home or dir == "/" then return nil end
        -- Walk up one level
        dir = dir:gsub("/[^/]*$", "")
        if dir == "" then dir = "/" end
    end
    return nil
end
```

### Merge

```lua
local function _merge_project_over_user(user_cfg, project_cfg, sources)
    -- Shallow merge: project top-level keys REPLACE user keys.
    -- Source-map tracks who set each key for :config show.
    for k, v in pairs(project_cfg) do
        user_cfg[k] = v
        sources[k] = "project"
    end
    -- (sources for unmodified user keys stay "user")
    return user_cfg
end
```

### Loader wrapper

```lua
local function load_config_with_overlay(opts)
    -- Existing load_config returns (user_cfg, user_path)
    local user_cfg, user_path = load_config(opts)

    local sources = {}
    for k, _ in pairs(user_cfg) do sources[k] = "user" end

    local proj_path = _find_project_config()
    if not proj_path then
        return user_cfg, sources, { user = user_path }
    end

    -- Trust check
    local trusted = _check_trusted(proj_path)
    if not trusted then
        if not _prompt_trust(proj_path) then
            -- declined; skip project layer
            return user_cfg, sources, { user = user_path, project = "(declined)" }
        end
    end

    local ok, proj_cfg = pcall(dofile, proj_path)
    if not ok or type(proj_cfg) ~= "table" then
        renderer.status("project config " .. proj_path .. " failed to load; ignoring")
        return user_cfg, sources, { user = user_path, project = "(load failed)" }
    end

    _merge_project_over_user(user_cfg, proj_cfg, sources)
    return user_cfg, sources, { user = user_path, project = proj_path }
end
```

Source map is then carried as a closure local in `repl.run` for `:config show`.

---

## 5. Pillar 3 — Trust prompt + persistent record

### Trust file shape

`~/.aish/trusted-projects` (mode 0600), JSONL:

```jsonl
{"path":"/home/user/src/aish/.aish.lua","sha256":"abc123...","ts":"2026-05-16T12:34:56Z"}
{"path":"/home/user/src/other/.aish.lua","sha256":"def456...","ts":"2026-05-16T12:40:00Z"}
```

### Trust check

```lua
local function _check_trusted(project_path)
    local path = (os.getenv("HOME") or "") .. "/.aish/trusted-projects"
    local f = io.open(path, "r")
    if not f then return false end
    local current_sha = _sha256_file(project_path)
    for line in f:lines() do
        local entry = json.decode(line)
        if entry and entry.path == project_path
                 and entry.sha256 == current_sha then
            f:close()
            return true
        end
    end
    f:close()
    return false
end
```

### Trust prompt

```lua
local function _prompt_trust(project_path)
    renderer.status("project config found: " .. project_path)
    renderer.status("UNTRUSTED. Loading it runs arbitrary Lua code.")
    local ans = rl.readline("[aish] trust this project config? [y/N] ")
    if ans and ans:lower():sub(1, 1) == "y" then
        _record_trust(project_path)
        return true
    end
    return false
end
```

### sha256

Shell out: `sha256sum <path> | cut -d' ' -f1`. POSIX-portable; faster than vendoring. Cached result during the trust check (single call per startup).

---

## 6. Pillar 4 — `:config show`

```
[aish] config sources:
  user:    ~/.config/aish/config.lua
  project: ~/src/aish/.aish.lua
[aish] effective config (top-level keys):
  default_model   : "fast"          (user)
  models          : {fast, cloud}   (project)
  shell           : {confirm_cmd=true, ...} (user)
  permissions     : {allow={...}, ...}  (project)
  hooks           : (unset)
  ...
```

Token-bearing fields (anything matching `auth_token`, `*_TOKEN`, etc.)
displayed as `(set)` rather than the value.

---

## 7. UX Surface Summary

| Meta | Behavior |
|---|---|
| `:config show` | Print resolved sources + sanitized effective config (read-only) |

| Startup status | Behavior |
|---|---|
| (no project file) | nothing — existing UX preserved |
| (project file found, untrusted) | `[aish] project config found: <path>` + `[aish] UNTRUSTED. Loading it runs arbitrary Lua.` + `[y/N]` prompt |
| (project file found, trusted, sha matches) | `[aish] project config: <path> (overlaid on <user>)` |
| (project file found, trusted, sha CHANGED) | re-prompt — bytes are different now |
| (declined this session) | `[aish] project config: <path> (declined this session)` |

No new config keys in v1 (the project overlay IS the new mechanism; it doesn't need a config flag to be enabled).

---

## 8. Out of Scope (Phase 9)

- **Sandboxed `.aish.lua` execution** — `dofile` runs full Lua; the
  trust prompt IS the gate. A sandbox (allowlisted globals,
  no `io.popen`, etc.) is bigger work and out of scope.
- **Reload on `cd`** — config is resolved at startup only. `cd`
  into a sibling project means restarting aish. Documented.
- **Recursive merge** — top-level shallow only.
- **Multiple project overlays** — walk-up stops at FIRST `.aish.lua`
  found. Nested projects (e.g., monorepo with per-package configs)
  would need deeper design; defer.
- **`:trust` / `:untrust` metas for runtime management** — trust
  records edited manually in `~/.aish/trusted-projects` for v1. A
  meta surface is a v2 polish.
- **Environment variable expansion in project file** — project file
  is plain Lua; users have `os.getenv` already.
- **Project-wide aish profile selection** — `.aish.lua` returns a
  config table, not a profile name. If multi-profile support is
  desired, the project file can compute a different config based
  on its OWN env vars / heuristics.

---

## 9. Risks

| Risk | Mitigation |
|---|---|
| Hostile `.aish.lua` in cloned repo runs arbitrary Lua on first `aish` run in that cwd | Trust prompt + sha256 persistence; default = decline if user just hits Enter at the [y/N]. |
| Trust file becomes corrupted / unreadable | Best-effort: corrupted lines skipped; missing file means all projects untrusted (re-prompt on next encounter). |
| User trusts `.aish.lua`, repo is updated, malicious code is injected | sha256 mismatch on next startup triggers re-prompt. User sees the prompt and can investigate before granting trust again. |
| `dofile` errors at load time (syntax error in project config) | pcall-protected; status line "project config X failed to load; ignoring" — aish continues with just the user config. |
| Walk-up walks above $HOME (e.g., a repo cloned to `/tmp`) | $HOME boundary check stops the walk. `/tmp` repos get no project layer (user can move them under $HOME or use --config). |
| Shallow merge surprises a user who wanted to add ONE model preset | Documented as predictable / explicit; users compose the full models block deliberately. |
| Source map dict grows unboundedly with new keys mid-session | Bounded by #config top-level keys (small constant; <20). No GC needed. |

---

## 10. Open Questions (Phase 9)

| # | Question | Impact | Resolution target |
|---|---|---|---|
| Q-P1 | Trust prompt before/after `aish: loaded config` status | A4 — **AFTER**; user sees user-config first, then decides about overlay. |
| Q-P2 | sha256 backend choice | B1 RESOLVED — `sha256sum` (GNU coreutils; universal on Linux); simpler output parsing than openssl. |
| Q-P3 | Log walk-up path | A5 — **no by default**; `:config show` reveals walk result on demand. Verbose-mode walk log is v2 polish. |
| Q-P4 | rl.readline safe at startup | A8 — DEFERRED to implement-time smoke (Phase 4 metas call rl.readline early too; new wrinkle is firing BEFORE main loop opens). If issue, fall back to printf+read shell-out. |
| Q-P5 | `:config show` full vs top-level | A6 — **top-level by default** (nested collapsed to inner keys); `:config show full` for deep dump. |
| Q-P6 | Project layer setting `secrets.vault` security | A7 — **allowed**; part of the trust prompt's scope. Bootstrap order (A12) ensures project's vault is honored if set. |

---

## 11. Phase 9 → Phase 10+ Out-of-band

Candidate follow-ups (non-binding):

- **Phase 10 candidates**:
  - Cost preflight enforcement (Phase 7 §12 option 2; Phase 8 §11 candidate).
  - Cross-session cost rollup (Phase 7 §12 option 1; Phase 8 §11 candidate).
  - `:trust` / `:untrust` metas for runtime trust management.
  - Sandboxed `.aish.lua` execution (allowlisted Lua globals).
- **Phase X+**: nested project overlays for monorepos; `:profile`
  switching; reload-on-cd.

Phase 9 itself is self-contained — depends on no specific prior phase
beyond the existing config loader.

---

## 13. Implementation Plan (commit-by-commit)

4 commits, bottom-up:

1. **`history.lua` — trust file helpers.**
   - `M.read_trusted(path)` -> list of `{path, sha256, ts}`
     entries; mode-check the file at 0600, refuse to load (warn)
     if wider. Missing file → empty list.
   - `M.add_trusted(trust_path, project_path, sha256)` appends a
     JSONL line; mkdir -p the parent if needed; chmod 0600.
   - `M.is_trusted(trust_path, project_path, sha256)` reads + checks
     for matching entry.
   - Internal `_sha256_file(path)` shells out to `sha256sum` and
     parses the first whitespace-separated field.
   - Smoke: 5 inline unit cases (read empty, add+read-back, mode
     check, sha mismatch returns false, missing file).

2. **`main.lua` — walk-up + load_with_project_overlay.**
   - `_find_project_config()` walks from libc.getcwd() up to $HOME,
     returning first `.aish.lua` or nil.
   - `_prompt_trust(project_path, sha)` calls `rl.readline` with
     the trust prompt; on accept, calls `history.add_trusted`.
     A8: smoke-test rl.readline behavior at this early call site.
   - `load_config_with_overlay(opts)` wraps existing `load_config`;
     finds project, checks trust, prompts if needed, dofiles +
     merges shallow over user config. Returns `(cfg, sources, paths)`
     triple where sources maps top-level keys to "user"/"project"
     for `:config show`.
   - `main()` calls `load_config_with_overlay`, stashes sources
     into a global so repl.lua can read it (or passes via cfg).
   - Smoke: tree-resolution test from a nested cwd; trust prompt
     accept/decline paths.

3. **`repl.lua` — `:config show` meta + startup status line.**
   - `:config show` / `:config show full` meta reads the sources
     map + the effective config; sanitizes token-bearing values
     (any key matching `_token`, `_TOKEN`, `auth_token` → display
     as `(set)`); prints sources + key-by-key effective.
   - Startup status line per A4: AFTER the existing `aish: loaded
     config from <path>`, if project layer fired, emit
     `[aish] project config: <path> (overlaid on <user>)`.
   - HELP gains 2 `:config` lines.
   - Smoke: with a test project file, run `:config show` and
     verify keys + sources line up.

4. **`config.lua` template note + status bump.**
   - Add a header comment to `config.lua` (the in-tree example)
     noting Phase 9 project-overlay availability (no other config
     change — overlay is a separate file).
   - PHASE9.md status header -> **Implement**.

### Risk index per commit

| Commit | Risk | Mitigation |
|---|---|---|
| 1 (history) | sha256sum not installed (some minimal images) | Detect at startup; if missing, warn + decline all trust prompts (project layer disabled). Documented. |
| 1 (history) | Trust file partial write (interrupted append) corrupts later parse | JSONL one-line-per-entry; partial line at EOF is skipped on read (each line is a single json.decode). |
| 2 (main) | A8 — rl.readline at startup (before main loop) untested in earlier phases | Smoke-test at commit-time; if broken, fall back to `io.read("*l")` from stdin (no readline frills like ^C-handling but functional). |
| 2 (main) | Walk-up symlink loops | `realpath`/`stat` defenses out of scope for v1; walk is bounded by $HOME stop. Pathological symlinks could waste cycles but not infinite-loop (every iteration strips a path component). |
| 3 (repl) | :config show might leak token values if a config key isn't matched by the masking heuristic | Conservative mask: any key containing "token", "secret", "auth", "key" (case-insensitive) → display `(set)`. Errs toward over-masking. |
| 4 (config + status) | None | |

### Tests + smoke per commit

Each commit:
- Pass `luajit test_safety.lua` (87/87) and `luajit test_router_model.lua` (31/31)
- Load cleanly via `luajit -e 'package.path=...; require("repl"); print("ok")'`
- Pass a per-feature smoke (described per row above)

### Things deliberately NOT split

- Separate `project.lua` module — small enough; history.lua already
  handles file-with-mode-check (memory.jsonl); same shape.
- :trust / :untrust runtime metas — manual ~/.aish/trusted-projects
  editing is fine for v1.
- Walk-up logging on first startup — easy to add later if needed.

### Open at plan-time (resolve at implement)

- A8: rl.readline early-startup behavior. If broken, swap to
  io.read("*l") for the trust prompt only.
- Whether to make the trust file path itself overridable via
  `$AISH_TRUST_FILE` env. Useful for CI / test isolation. Default
  to ~/.aish/trusted-projects; env override is one line.