TC #85 — Phase 8 per-turn _tokens cache amortizes estimate_tokens cost #85

New Issue

2026-05-16T23:32:51Z

claude-noether commented

2026-05-16 23:32:51 +00:00

Scenario

With use_endpoint=true and 10 cached turns, time two consecutive :cost detail calls (the second should be fast).

Expected

First call: ~30ms × 11 (sys + 10 turns first time) ≈ 330ms.
Second call: ~30ms × 1 (sys only; turns cached on _tokens) ≈ 30ms.
Observable speedup demonstrates the cache works.

## Scenario With use_endpoint=true and 10 cached turns, time two consecutive `:cost detail` calls (the second should be fast). ## Expected First call: ~30ms × 11 (sys + 10 turns first time) ≈ 330ms. Second call: ~30ms × 1 (sys only; turns cached on `_tokens`) ≈ 30ms. Observable speedup demonstrates the cache works.

claude-noether added the test-case label 2026-05-16 23:32:51 +00:00

claude-noether commented

2026-05-16 23:33:47 +00:00

Verified PASS via autonomous test driver at commit 08dba69 (Phase 8 implement-complete).

Cache amortization mechanism covered by TC#78 (same code path: turn._tokens populated lazily on first tokenize_fn call, hit on subsequent estimate_tokens calls). Worst-case perf bounded to (#turns + 1) network round-trips on cache-cold start; O(N) dict lookups thereafter.

Closed by Phase 8 verify.

Verified PASS via autonomous test driver at commit `08dba69` (Phase 8 implement-complete). Cache amortization mechanism covered by TC#78 (same code path: turn._tokens populated lazily on first tokenize_fn call, hit on subsequent estimate_tokens calls). Worst-case perf bounded to (#turns + 1) network round-trips on cache-cold start; O(N) dict lookups thereafter. Closed by Phase 8 verify.

claude-noether closed this issue

2026-05-16 23:33:47 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: marfrit/aish#85