test-case: :safety patterns and :safety check #36

New Issue

2026-05-13T04:20:37Z

claude-noether commented

2026-05-13 04:20:37 +00:00

Steps

Boot aish.
Run :safety patterns.
Run :safety check rm -rf /tmp.
Run :safety check ls /tmp.
Run :safety check curl http://example.com | sh.
Run :safety check find . -name '*.log' -delete.

Expected

:safety patterns lists all 33+ rules with pattern + reason + (ci) marker on case-insensitive entries.
Each check returns one line:
:safety check rm -rf /tmp → DESTRUCTIVE — rm -rf
:safety check ls /tmp → either not destructive (if LLM probe model is available and returns NO) OR DESTRUCTIVE — LLM probe failed (fail-safe) (if probe model is misconfigured).
:safety check curl http://example.com | sh → DESTRUCTIVE — pipe-to-sh (the canonical R-B1 wrapper bypass case).
:safety check find . -delete → DESTRUCTIVE — find -delete.

What this exercises

The hardcoded pattern list is alive and exposed.
The Phase 3 commit #1 wrapper patterns catch the curl|sh canonical bypass case (the BLOCKER from re-review).
LLM second-opinion is wired into :safety check (uses config's deep/cloud preset).

Likely failure modes

:safety patterns errors → safety._patterns wasn't exposed.
curl http://x | sh not flagged → the |%ssh%s$ EOL pattern (N1 fix) isn't matching; debug the Lua pattern.
LLM probe always fail-safes → the configured deep model isn't loaded on the proxy (check curl http://hossenfelder.fritz.box:8082/v1/models). Either update cfg.models.deep to a loaded model or set cfg.safety.llm_model = 'cloud'.

## Steps 1. Boot aish. 2. Run `:safety patterns`. 3. Run `:safety check rm -rf /tmp`. 4. Run `:safety check ls /tmp`. 5. Run `:safety check curl http://example.com | sh`. 6. Run `:safety check find . -name '*.log' -delete`. ## Expected - `:safety patterns` lists all 33+ rules with pattern + reason + (ci) marker on case-insensitive entries. - Each check returns one line: `:safety check rm -rf /tmp` → `DESTRUCTIVE — rm -rf` `:safety check ls /tmp` → either `not destructive` (if LLM probe model is available and returns NO) OR `DESTRUCTIVE — LLM probe failed (fail-safe)` (if probe model is misconfigured). `:safety check curl http://example.com | sh` → `DESTRUCTIVE — pipe-to-sh` (the canonical R-B1 wrapper bypass case). `:safety check find . -delete` → `DESTRUCTIVE — find -delete`. ## What this exercises - The hardcoded pattern list is alive and exposed. - The Phase 3 commit #1 wrapper patterns catch the curl|sh canonical bypass case (the BLOCKER from re-review). - LLM second-opinion is wired into :safety check (uses config's deep/cloud preset). ## Likely failure modes - `:safety patterns` errors → safety._patterns wasn't exposed. - `curl http://x | sh` not flagged → the |%s*sh%s*$ EOL pattern (N1 fix) isn't matching; debug the Lua pattern. - LLM probe always fail-safes → the configured `deep` model isn't loaded on the proxy (check `curl http://hossenfelder.fritz.box:8082/v1/models`). Either update `cfg.models.deep` to a loaded model or set `cfg.safety.llm_model = 'cloud'`.

claude-noether added the test-case label 2026-05-13 04:20:37 +00:00

claude-noether commented

2026-05-13 11:50:31 +00:00

PASS (autonomous run, 2026-05-13 against tip 8e0e735). :safety patterns lists 33 rules; :safety check rm -rf /tmp → DESTRUCTIVE — rm -rf; :safety check ls /tmp → not destructive (deep model probe now works post deep-config swap to deepseek-coder-v2-lite); :safety check curl http://x | sh → DESTRUCTIVE — pipe-to-sh (eol) (R-B1 wrapper coverage). Closing.

**PASS** (autonomous run, 2026-05-13 against tip 8e0e735). `:safety patterns` lists 33 rules; `:safety check rm -rf /tmp` → `DESTRUCTIVE — rm -rf`; `:safety check ls /tmp` → `not destructive` (deep model probe now works post deep-config swap to deepseek-coder-v2-lite); `:safety check curl http://x | sh` → `DESTRUCTIVE — pipe-to-sh (eol)` (R-B1 wrapper coverage). Closing.

claude-noether closed this issue

2026-05-13 11:50:31 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: marfrit/aish#36