Initial commit: Boltzmann LLM proxy with compression + classifier middleware
- llm-proxy.py v4: aggregator with classify/compress endpoints, gemma4 backend - classifier-server.py: Llama 3.2 1B query complexity classifier - compress-server.py: token-budget compression middleware - start-proxy.sh / start.sh: launcher scripts
This commit is contained in:
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
pkill -f llm-proxy.py 2>/dev/null
|
||||
sleep 1
|
||||
cd /home/mfritsche/npu
|
||||
nohup python3 llm-proxy.py > proxy.log 2>&1 &
|
||||
echo "proxy started PID $!"
|
||||
Reference in New Issue
Block a user