From 8d90c0c277d82ded0ae25d779a553aa134535340 Mon Sep 17 00:00:00 2001 From: Kagamine Rin Date: Thu, 4 Jun 2026 21:24:41 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20update=20README=20and=20ARCHITECTURE=20?= =?UTF-8?q?for=20v1.1.0=20=E2=80=94=20litellm=5Fhistory=20function?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README: three functions, litellm_history params and cost awareness note - ARCHITECTURE: full Function 3 section with endpoint, fields, output format, token display rules, error paths, and performance considerations --- ARCHITECTURE.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++-- README.md | 13 ++++++++-- 2 files changed, 74 insertions(+), 4 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 31b1eb1..9ef5517 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -5,7 +5,7 @@ ## Product Definition -A single Open WebUI **Tool** that provides LiteLLM proxy health status and cost breakdown. Designed for the Network but built for general use — any LiteLLM proxy instance. +A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost breakdown, and recent request history. Designed for the Network but built for general use — any LiteLLM proxy instance. ## OWI Integration Model @@ -157,6 +157,67 @@ By Date: | Timeout (>30s) | Timeout message (large date ranges may be slow) | | No results | "No cost data found for [range]" | +## Function 3: Request History + +**LiteLLM Endpoint:** `GET {litellm_url}/spend/logs` +**Auth:** `Authorization: Bearer {litellm_master_key}` +**Params:** `limit` (int, 1-50, default 20), `show_cost` (bool, default true) + +### Response Processing + +The `/spend/logs` endpoint returns **all** historical request logs (no server-side pagination). The tool sorts by `startTime` descending and takes the most recent `limit` entries client-side. + +**Performance note:** For large deployments (thousands of logs), this endpoint can be slow. The tool caps `limit` at 50 and uses a 30s timeout. + +### Per-Request Fields + +| Field | Description | +|-------|-------------| +| `request_id` | Unique request identifier | +| `model` | Model used (provider prefix stripped for readability) | +| `call_type` | Request type: `acompletion` → "chat", `aembedding` → "embed", `aimage_generation` → "image" | +| `status` | ✅ OK if no exception, ❌ FAIL if exception present | +| `startTime` | Request start timestamp | +| `request_duration_ms` | Total duration | +| `prompt_tokens` | Input tokens | +| `completion_tokens` | Output tokens | +| `cache_read_input_tokens` | Cache hit count | +| `spend` | Cost in USD | +| `metadata.exception` | Error message if failed | + +### Output Format + +``` +LiteLLM Request History — http://host.docker.internal:4000 +Showing 20 of 4560 total requests (most recent first) +========================================================================================== + 1. 06-05T01:15:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 15.0s | 96→1.6k | $0.0947 + 2. 06-05T01:12:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 13.3s | 101→1.5k | $0.0922 + 3. 06-05T01:11:35 | | | ✅ OK | 0ms + 4. 06-05T01:07:36 | gemini-3.5-flash | chat | ✅ OK | 8.8s | 149.8k→1.1k | $0.1821 + ... + + Summary (20 shown): $3.1493 spent | 2,798,587 prompt + 5,815 completion tokens | 0 failed +``` + +### Token Display Rules + +- Prompt tokens ≥ 1000: shown as "X.Xk" (e.g., 149.8k) +- Completion tokens ≥ 1000: shown as "X.Xk" +- Cache hit % shown as 💾 indicator when `cache_read_input_tokens > 0` +- Zero-token entries (health checks, failed routing) shown without token info + +### Error Paths + +| Condition | Response | +|-----------|----------| +| No master key configured | Error — key required | +| 401 Unauthorized | Auth failed message | +| 403 Forbidden | Permission denied message | +| Connection refused | Connection failed message | +| Timeout (>30s) | Timeout message | +| No logs | "No request history found" | + ## Additional Endpoints (Future Consideration) | Endpoint | Purpose | Priority | @@ -169,7 +230,7 @@ These are documented but not in scope for v1. The two-function design is the rig ## Design Principles -1. **Two functions, that's it.** No scope creep. Health check and cost breakdown. +1. **Three functions, that's it.** No scope creep. Health check, cost breakdown, request history. 2. **Portable.** No hardcoded URLs or keys. Valves pass everything. 3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps. 4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy. diff --git a/README.md b/README.md index 3214cbe..645750e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # LiteLLM Dashboard — Open WebUI Tool -A simple, two-function Open WebUI Tool for checking LiteLLM proxy health and cost breakdowns. +A simple, three-function Open WebUI Tool for checking LiteLLM proxy health, cost breakdowns, and recent request history. ## Functions @@ -10,6 +10,15 @@ Check LiteLLM proxy health and authentication status. Returns which models are h ### `litellm_cost` Get LiteLLM cost breakdown for a date range. Returns per-model and per-day spend, token counts, cache hit rate, and request stats. +### `litellm_history` +Get recent LiteLLM request history. Returns the most recent N requests with model, call type, status, duration, token counts, cache hit %, and per-request cost. Useful for understanding which models are being called and how much individual requests cost. + +**Parameters:** +- `limit` (int, 1-50, default 20): Number of recent requests to show +- `show_cost` (bool, default true): Include per-request cost + +**Cost awareness:** This function hits `/spend/logs` which returns all historical logs. For large deployments, this can be a heavy query. Use reasonable `limit` values. + ## Valves (Configuration) | Valve | Default | Purpose | @@ -33,7 +42,7 @@ Both valves are passed at runtime — no hardcoded keys. The tool is portable to ## Design Principles -- **Two functions, that's it.** No scope creep. +- **Three functions, that's it.** No scope creep. - **Portable.** No hardcoded URLs or keys. - **Token-efficient.** Strips `litellm_metadata` (~2KB/model) from health responses. Formats cost data as readable text, not raw JSON dumps. - **General-purpose.** Works with any LiteLLM proxy instance.