docs: update README and ARCHITECTURE for v1.1.0 — litellm_history function

- README: three functions, litellm_history params and cost awareness note
- ARCHITECTURE: full Function 3 section with endpoint, fields, output format,
  token display rules, error paths, and performance considerations
This commit is contained in:
rin.kagamine committed 2026-06-04 21:24:41 -04:00
1 parent fd449079be
commit 8d90c0c277
2 files changed
+74 -4

No files matched your search

+63 -2
View File
@@ -5,7 +5,7 @@
## Product Definition
A single Open WebUI **Tool** that provides LiteLLM proxy health status and cost breakdown. Designed for the Network but built for general use — any LiteLLM proxy instance.
A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost breakdown, and recent request history. Designed for the Network but built for general use — any LiteLLM proxy instance.
## OWI Integration Model
@@ -157,6 +157,67 @@ By Date:
| Timeout (>30s) | Timeout message (large date ranges may be slow) |
| No results | "No cost data found for [range]" |
## Function 3: Request History
**LiteLLM Endpoint:** `GET {litellm_url}/spend/logs`
**Auth:** `Authorization: Bearer {litellm_master_key}`
**Params:** `limit` (int, 1-50, default 20), `show_cost` (bool, default true)
### Response Processing
The `/spend/logs` endpoint returns **all** historical request logs (no server-side pagination). The tool sorts by `startTime` descending and takes the most recent `limit` entries client-side.
**Performance note:** For large deployments (thousands of logs), this endpoint can be slow. The tool caps `limit` at 50 and uses a 30s timeout.
### Per-Request Fields
| Field | Description |
|-------|-------------|
| `request_id` | Unique request identifier |
| `model` | Model used (provider prefix stripped for readability) |
| `call_type` | Request type: `acompletion` → "chat", `aembedding` → "embed", `aimage_generation` → "image" |
| `status` | ✅ OK if no exception, ❌ FAIL if exception present |
| `startTime` | Request start timestamp |
| `request_duration_ms` | Total duration |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `cache_read_input_tokens` | Cache hit count |
| `spend` | Cost in USD |
| `metadata.exception` | Error message if failed |
### Output Format
```
LiteLLM Request History — http://host.docker.internal:4000
Showing 20 of 4560 total requests (most recent first)
==========================================================================================
1. 06-05T01:15:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 15.0s | 96→1.6k | $0.0947
2. 06-05T01:12:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 13.3s | 101→1.5k | $0.0922
3. 06-05T01:11:35 | | | ✅ OK | 0ms
4. 06-05T01:07:36 | gemini-3.5-flash | chat | ✅ OK | 8.8s | 149.8k→1.1k | $0.1821
...
Summary (20 shown): $3.1493 spent | 2,798,587 prompt + 5,815 completion tokens | 0 failed
```
### Token Display Rules
- Prompt tokens ≥ 1000: shown as "X.Xk" (e.g., 149.8k)
- Completion tokens ≥ 1000: shown as "X.Xk"
- Cache hit % shown as 💾 indicator when `cache_read_input_tokens > 0`
- Zero-token entries (health checks, failed routing) shown without token info
### Error Paths
| Condition | Response |
|-----------|----------|
| No master key configured | Error — key required |
| 401 Unauthorized | Auth failed message |
| 403 Forbidden | Permission denied message |
| Connection refused | Connection failed message |
| Timeout (>30s) | Timeout message |
| No logs | "No request history found" |
## Additional Endpoints (Future Consideration)
| Endpoint | Purpose | Priority |
@@ -169,7 +230,7 @@ These are documented but not in scope for v1. The two-function design is the rig
## Design Principles
1. **Two functions, that's it.** No scope creep. Health check and cost breakdown.
1. **Three functions, that's it.** No scope creep. Health check, cost breakdown, request history.
2. **Portable.** No hardcoded URLs or keys. Valves pass everything.
3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps.
4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy.
+11 -2
View File
@@ -1,6 +1,6 @@
# LiteLLM Dashboard — Open WebUI Tool
A simple, two-function Open WebUI Tool for checking LiteLLM proxy health and cost breakdowns.
A simple, three-function Open WebUI Tool for checking LiteLLM proxy health, cost breakdowns, and recent request history.
## Functions
@@ -10,6 +10,15 @@ Check LiteLLM proxy health and authentication status. Returns which models are h
### `litellm_cost`
Get LiteLLM cost breakdown for a date range. Returns per-model and per-day spend, token counts, cache hit rate, and request stats.
### `litellm_history`
Get recent LiteLLM request history. Returns the most recent N requests with model, call type, status, duration, token counts, cache hit %, and per-request cost. Useful for understanding which models are being called and how much individual requests cost.
**Parameters:**
- `limit` (int, 1-50, default 20): Number of recent requests to show
- `show_cost` (bool, default true): Include per-request cost
**Cost awareness:** This function hits `/spend/logs` which returns all historical logs. For large deployments, this can be a heavy query. Use reasonable `limit` values.
## Valves (Configuration)
| Valve | Default | Purpose |
@@ -33,7 +42,7 @@ Both valves are passed at runtime — no hardcoded keys. The tool is portable to
## Design Principles
- **Two functions, that's it.** No scope creep.
- **Three functions, that's it.** No scope creep.
- **Portable.** No hardcoded URLs or keys.
- **Token-efficient.** Strips `litellm_metadata` (~2KB/model) from health responses. Formats cost data as readable text, not raw JSON dumps.
- **General-purpose.** Works with any LiteLLM proxy instance.