llmproxy

2 Commits 2 Branches 0 Tags

Author	SHA1	Message	Date
Markus Fritsche	ac77f054c2	systemd: add llama-server-qwen3-4b-npu.service (NPU Qwen3-4B on :8091) Boot-persistent rk-llama.cpp rknpu2 backend for qwen3-4b-2507-npu. Kernel gate matches any npu kernel (was pinned to the retired 6.1.75-npu-port; current boltzmann kernel is 7.0.0-rc3-npuclk+). LimitNOFILE=65536 for librknnrt dmabuf fds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE	2026-07-19 17:57:39 +02:00
Markus Fritsche	c121428c58	Initial commit: Boltzmann LLM proxy with compression + classifier middleware - llm-proxy.py v4: aggregator with classify/compress endpoints, gemma4 backend - classifier-server.py: Llama 3.2 1B query complexity classifier - compress-server.py: token-budget compression middleware - start-proxy.sh / start.sh: launcher scripts	2026-06-15 15:12:25 +02:00

Author

SHA1

Message

Date

Markus Fritsche

ac77f054c2

systemd: add llama-server-qwen3-4b-npu.service (NPU Qwen3-4B on :8091)

Boot-persistent rk-llama.cpp rknpu2 backend for qwen3-4b-2507-npu. Kernel
gate matches any *npu* kernel (was pinned to the retired 6.1.75-npu-port;
current boltzmann kernel is 7.0.0-rc3-npuclk+). LimitNOFILE=65536 for
librknnrt dmabuf fds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE

2026-07-19 17:57:39 +02:00

Markus Fritsche

c121428c58

Initial commit: Boltzmann LLM proxy with compression + classifier middleware

- llm-proxy.py v4: aggregator with classify/compress endpoints, gemma4 backend
- classifier-server.py: Llama 3.2 1B query complexity classifier
- compress-server.py: token-budget compression middleware
- start-proxy.sh / start.sh: launcher scripts

2026-06-15 15:12:25 +02:00