test-case: max_tool_depth cap and orphan-tool_calls protection #25
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: marfrit/aish#25
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Steps
mcp.max_tool_depth = 2for a tight cap.Call boltzmann.list_dir on /tmp, then on /home, then on /etc.— try to encourage multiple sequential calls.Expected
[aish] tool-call depth limit reached (2); stopping sub-loop.:historyshows: user → assistant(tool_calls) → tool → assistant(tool_calls) → tool → (no more iterations).tool_callsin context — every assistant-with-tool_calls turn is followed by its matching tool turn(s).What this exercises
max_tool_depthenforcementLikely failure modes
mfritsche aish main ≡ ~1 luajit main.lua
aish: loaded config from ./config.lua
[aish] mcp boltzmann: 7 tools
[aish:fast]> Call boltzmann.list_dir on /tmp, then on /home, then on /etc.
CMD: boltzmann.list_dir /tmp
CMD: boltzmann.list_dir /home
CMD: boltzmann.list_dir /etc
execute 'boltzmann.list_dir /tmp'? [y/N] y
─── exec output ───
/bin/sh: line 1: boltzmann.list_dir: command not found
─── exit 127 ───
execute 'boltzmann.list_dir /home'? [y/N] y
─── exec output ───
/bin/sh: line 1: boltzmann.list_dir: command not found
─── exit 127 ───
execute 'boltzmann.list_dir /etc'? [y/N] y
─── exec output ───
/bin/sh: line 1: boltzmann.list_dir: command not found
─── exit 127 ───
probably need a tighter prompt for the fast model
execution:
[aish:deep]>
[aish:deep]> without using CMD: lines, call boltzmann.list_dir three times via tool_call: first with path /tmp, then /home, then /etc.
Okay, the user wants me to call the boltzmann.list_dir function three times with different paths: /tmp, /home, and /etc. Let me check the tools provided.
First, I need to make sure that the function name is correct. The tool is listed as boltzmann.list_dir, and the parameters require a path, which defaults to "." if not specified. So for each directory, I'll set the path parameter accordingly.
The user specified not to use CMD: lines, so I must use tool_call for each invocation. Each call should be a separate tool_call block. Let me structure each one properly. The first call is to /tmp, then /home, then /etc. I'll make sure the JSON syntax is correct for each, with the "name" as "boltzmann.list_dir" and the "arguments" containing the path. No other parameters are needed since the others have defaults. Alright, that should do it.
call 'boltzmann.list_dir({"path":"/tmp"})'? [y/N] y
─── tool: boltzmann.list_dir ───
{"path": "/tmp"}
boltzmann-aggregator.log
chat-out2.json
chat-out.json
checkup-db-0
distccd.log
expect-e2.cap
harness-probe.sh
interact-bridge-test.sh
interact-test.cap
llm-proxy-hossenfelder.py
lua_13s0rA
lua_64ekpD
lua_6w9H0D
lua_73626k
lua_7T7qBS
lua_8hi5Vq
lua_99llaN
lua_A0QhdK
lua_ARwOTZ
lua_axC2Xy
lua_axC2Xy.out
lua_bKjM5r
lua_cB6eIu
lua_FCPyTb
lua_FLcMJT
lua_gKMDnQ
lua_GmpGJ6
lua_gSCLUV
lua_hp9IPf
lua_iCPpn7
lua_Idr4eL
lua_IgMTkJ
lua_iLGoXc
lua_IXL9BI
lua_k8c3Gv
lua_KW7P9W
lua_LPOnKV
lua_m0Ofyk
lua_MpXkMw
lua_NbNBEy
lua_NDsuSz
lua_nLYpy1
lua_nX4uGL
lua_ofGB27
lua_oFmUck
lua_oY7okS
lua_PHerfK
lua_pXg221
lua_qaAlHG
lua_QewwBN
lua_qIIRtU
lua_qNKqky
lua_qORYaF
lua_qu5jr5
lua_r0Nzla
lua_S6hDrp
lua_SaSn9u
lua_sKm2pt
lua_SufqG3
lua_TnhaO6
lua_Ufloiq
lua_vBnBhV
lua_VcRyFV
lua_w1SLbY
lua_wj6TsZ
lua_XeA1wF
lua_yDzlGI
lua_YZJc9j
lua_ZzJl6x
news_cache.json
news_run_1000.txt
probe2.log
probe.log
probe-result.log
qwen3-server.log
restart-hossenfelder-proxy.sh
rfc_v2_series.patch
r.json
rkvdec_consumer.patch
sddm-:0-OXXwiH
sddm-auth-62871827-9199-416e-80cf-8096a6703a2e
stream-bg.log
stream.end
stream.raw
stream.start
stream-trace.log
sysprompt.txt
systemd-private-a5fb73a5fb7e4b748bc0450d418af012-bluetooth.service-wkVHKh
systemd-private-a5fb73a5fb7e4b748bc0450d418af012-polkit.service-BSOo3j
systemd-private-a5fb73a5fb7e4b748bc0450d418af012-systemd-logind.service-TkPhwG
systemd-private-a5fb73a5fb7e4b748bc0450d418af012-upower.service-Y9ttmV
t4.cap
t4-wrap.cap
t5.cap
t-bridge.cap
test-A.cap
test-B.cap
test-bridge.sh
test-interact.sh
wrap2.cap
wrap.cap
xauth_wmBukK
─── ok ───
call 'boltzmann.list_dir({"path":"/home"})'? [y/N] y
─── tool: boltzmann.list_dir ───
{"path": "/home"}
build
mfritsche
─── ok ───
call 'boltzmann.list_dir({"path":"/etc"})'? [y/N] y
─── tool: boltzmann.list_dir ───
{"path": "/etc"}
alsa
apparmor.d
arch-release
arptables.conf
audisp
audit
avahi
bash.bash_logout
bash.bashrc
bash_completion.d
bindresvport.blacklist
binfmt.d
bluetooth
bredos-dracut.conf
ca-certificates
cifs-utils
colord
conf.d
credstore
credstore.encrypted
cryptsetup-keys.d
crypttab
cups
daxctl.conf.d
dconf
debuginfod
default
depmod.d
distcc
dnsmasq.conf
dracut.conf
dracut.conf.d
e2scrub.conf
ebtables.conf
environment
environment.pacnew
ethertypes
fonts
freetds
fstab
fuse.conf
gai.conf
gdb
gnutls
govctl
gprofng.rc
group
group-
group.pacnew
grub.d
gshadow
gshadow-
gshadow.pacnew
gssapi_mech.conf
gtk-3.0
healthd.conf
host.conf
hostname
hosts
ImageMagick-7
infiniband-diags
inputrc
iptables
issue
iwpmd.conf
java-openjdk
kernel
keyutils
krb5.conf
ld.so.cache
ld.so.conf
ld.so.conf.d
libao.conf
libaudit.conf
libblockdev
libibverbs.d
libinput
libnl
libpaper.d
libva.conf
libvirt
lightdm
locale.conf
locale.gen
locale.gen.pacnew
localtime
login.defs
logrotate.d
lsb-release
lxc
machine-id
mailcap
makepkg.conf
makepkg.conf.d
makepkg.conf.pacnew
makepkg.d
man_db.conf
mc
mdadm.conf
mercurial
mime.types
mke2fs.conf
mkinitcpio.conf
mkinitcpio.conf.bak
mkinitcpio.d
ModemManager
modprobe.d
modules-load.d
mtab
mtools.conf
nanorc
ndctl
ndctl.conf.d
netconfig
NetworkManager
nftables.conf
nginx
nilfs_cleanerd.conf
nohang
nsswitch.conf
ODBCDataSources
odbc.ini
odbcinst.ini
OpenCL
openldap
os-release
pacman.conf
pacman.conf.bak
pacman.conf.pacnew
pacman.d
pam.d
papersize
paperspecs
passwd
passwd-
passwd.pacnew
pipewire
pkcs11
polkit-1
ppp
profile
profile.d
protocols
pulse
qemu
rc_keymaps
rc_maps.cfg
rdma
request-key.conf
request-key.d
resolv.conf
rhashrc
rpc
rsyncd.conf
sasl2
screenrc
sddm.conf
securetty
security
sensors3.conf
sensors.d
services
shadow
shadow-
shadow.pacnew
shells
signond.conf
skel
slsh.rc
smartd.conf
srp_daemon.conf
ssh
ssl
subgid
subgid-
subuid
subuid-
sudo.conf
sudoers
sudoers.d
sudo_logsrvd.conf
sysctl.d
systemd
tmpfiles.d
tpm2-tss
ts.conf
u-boot-menu
udev
udisks2
UPower
user-tmpfiles.d
vconsole.conf
vde2
vdpau_wrapper.cfg
vulkan
wgetrc
wpa_supplicant
X11
xattr.conf
xdg
xml
─── ok ───
Okay, let me see what the user is asking for here. They previously wanted three list_dir calls for /tmp, /home, and /etc. The assistant made those tool calls, and the responses are shown. Now, the user hasn't asked a new question, but the assistant needs to respond based on the provided tool responses.
The tool responses are the directory listings for each path. The user might be expecting an summary or confirmation of these listings. Since the user's last instruction was to make the three tool calls, and the responses are already provided, the assistant should probably just confirm that the operations were completed or ask if there's anything else needed.
But looking at the history, the user's last message was the tool responses, and the assistant needs to generate a response. The user might be testing if the assistant can process the results. However, since the original request was just to make the three calls, and the assistant did that, the next step is to inform the user that the directory listings have been retrieved and offer further assistance.
So the appropriate response would be to state that the three directory listings were successfully retrieved and ask if there's anything more the user needs. That's concise and helpful without overcomplicating things.
The three directory listings have been retrieved successfully. Let me know if you need further assistance!
[aish:deep]>
PASS for multi-call dispatch; cap path not actually exercised.
The :model deep trace shows the sub-loop dispatching three sequential tool calls correctly: confirm gate fired for each, each rendered as its own
─── tool: ... ───frame, each result appended as a role:"tool" turn, and the model wrapped up with a final pure-text response. End-to-end sub-loop is sound.Caveat on the cap path:
max_tool_depthcounts sub-loop iterations, not tool_calls within a single iteration. The deep model emitted all three tool_calls in one assistant response (a singlefinish_reason: tool_calls), so the dispatcher ran one iteration that processed all three calls — depth = 1. To force the cap, the model would have to emit one tool_call per response, receive the result, then emit another, etc. That's a different model-behavior axis (chain-of-tools vs. parallel-tools) and harder to provoke without explicit prompting.The cap logic itself was unit-tested during commit #6 via the mocked-broker test (verified the break + status emission), so confidence in the code is high; the live exercise is just deferred to whenever a model decides to chain rather than parallel-call.
Closing as PASS-on-substance.