Initial commit: llm-proxy (de-hardcoded, portable)

Zero-dependency OpenAI-compatible router: OpenRouter cloud + local model-aware
fanout, merged /v1/models catalog with cost tags, background free/cheap-tier
auto-curation. Extracted from the fleet's unversioned /opt/llm-proxy.py (kept
only as cp-backups) and de-hardcoded for portability:
- config dir via LLM_PROXY_CONFIG_DIR (default /etc/llm-proxy)
- default backends -> localhost (no fleet hosts); real ones in local-backends.json
- friendly-name aliases -> friendly-names.json (none baked in code)
- OpenRouter Referer/Title via LLM_PROXY_REFERER / LLM_PROXY_TITLE
- removed the ds4-escher one-off filter and the fleet-specific helper scripts
Sanitized: no fleet hostnames, IPs, names, or secrets (key comes from env).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWpfhDgYNA21tETDP9ueBE

.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.pyc
+*.log
+*.bak
+*.bak-*
+# real config + runtime state (only *.example.json is committed)
+local-backends.json
+friendly-names.json
+cloud-catalog.json
+last-known-models.json

README.md

@@ -0,0 +1,101 @@
+# llm-proxy
+
+An OpenAI-compatible HTTP proxy that routes `/v1` requests to either OpenRouter
+(cloud) or local model servers. One file, Python standard library only, no
+dependencies.
+
+## What it does
+
+- Presents a single OpenAI-compatible `/v1` API in front of many backends.
+- Routes each request by model id: cloud-prefixed ids go to OpenRouter; other
+  ids go to the local backend that advertises them.
+- Serves a merged `/v1/models` list — the OpenRouter catalog plus every
+  reachable local server's models — with each id tagged by source and cost tier.
+- Maintains the cloud catalog in the background: adds newly-free models, drops
+  models that stopped being free, marks unavailable ones, and discovers cheap
+  paid models.
+
+## Requirements
+
+- Python 3, standard library only.
+- `OPENROUTER_API_KEY` in the environment for cloud routing. Without it, only
+  local routing works.
+
+## Run
+
+    OPENROUTER_API_KEY=sk-or-... python3 llm-proxy.py 8082
+
+The listen port is the first argument (default `8082`). The server binds `0.0.0.0`.
+
+## Configuration
+
+### Environment
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `OPENROUTER_API_KEY` | — | Bearer key for OpenRouter. Substituted server-side; clients never send it. |
+| `LLM_PROXY_CONFIG_DIR` | `/etc/llm-proxy` | Directory holding the files below. |
+| `LLM_PROXY_REFERER` | `http://localhost` | `HTTP-Referer` sent to OpenRouter. |
+| `LLM_PROXY_TITLE` | `llm-proxy` | `X-Title` sent to OpenRouter. |
+
+### Files (in `LLM_PROXY_CONFIG_DIR`)
+
+| File | In repo | Purpose |
+|---|---|---|
+| `local-backends.json` | example only | Local backends: `{name, host, port, slots}`. Copy `local-backends.example.json`. |
+| `friendly-names.json` | example only | Optional display aliases `{model-id: label}`. Copy `friendly-names.example.json`. |
+| `cloud-catalog.json` | no (state) | Persisted OpenRouter catalog + status. Written by the proxy. |
+| `last-known-models.json` | no (state) | Last-seen model list. Written by the proxy. |
+
+If `local-backends.json` is absent, the proxy falls back to one `127.0.0.1:8080`
+backend.
+
+## Endpoints
+
+- `POST /v1/chat/completions` — routed by model id (see Routing); streaming supported.
+- `GET /v1/models` — merged local + cloud catalog.
+
+Resolved requests are forwarded to the chosen backend and the response streamed back.
+
+## Routing
+
+1. A **cloud-prefixed** model id (`openai/`, `anthropic/`, `google/`,
+   `deepseek/`, `qwen/`, `x-ai/`, …) goes to OpenRouter over HTTPS with the
+   server-side key.
+2. A **known local** model goes to the backend whose `/v1/models` advertises it
+   (cached 30 s).
+3. Unknown model or empty body: the first reachable local backend.
+
+The `/v1/models` ids carry display tags — `[$]` paid cloud, `[free]` free cloud,
+`[local]` local. The tag is stripped before routing, so a client may send back
+the exact id it was shown.
+
+## Background catalog maintenance
+
+A thread runs every 3 hours and:
+
+- adds newly-free models over 7 B parameters to the free catalog,
+- removes models that are no longer free,
+- probes each free model with a 1-token request and suffixes `-na` on HTTP 429,
+- discovers paid models under `$0.50` / M-token from known prefixes.
+
+Paid models added by hand in `cloud-catalog.json` are never auto-removed.
+
+## Signals
+
+- `SIGHUP` — reload `local-backends.json`.
+- `SIGUSR1` — drop discovery caches and re-check the cloud catalog now.
+
+## Scope
+
+llm-proxy has no virtual keys, budgets, caching, observability callbacks, admin
+UI, or native non-OpenRouter providers. It trades those for a single
+dependency-free file plus OpenRouter free/cheap-tier auto-curation. For the
+larger feature set, use a full gateway such as litellm.
+
+## bullpen
+
+Contributed as an optional component of
+[bullpen](https://git.reauktion.de/marfrit/bullpen). bullpen depends only on a
+generic OpenAI-compatible `/v1` endpoint; llm-proxy is one way to provide it,
+not a requirement.

friendly-names.example.json

@@ -0,0 +1,4 @@
+{
+  "some-provider/very-long-model-id-2507": "short-label",
+  "another-local-model-q4-k-m": "another"
+}

local-backends.example.json

@@ -0,0 +1,4 @@
+[
+  {"name": "local", "host": "127.0.0.1", "port": 8080, "slots": 1},
+  {"name": "gpu-host", "host": "gpu-host.example.lan", "port": 8080, "slots": 4}
+]