docs: update README and ARCHITECTURE for v1.1.0 — litellm_history function
- README: three functions, litellm_history params and cost awareness note - ARCHITECTURE: full Function 3 section with endpoint, fields, output format, token display rules, error paths, and performance considerations
This commit is contained in:
1 parent
fd449079be
commit
8d90c0c277
2 files changed
+74
-4
No files matched your search
+63
-2
@@ -5,7 +5,7 @@
|
|||||||
|
|
||||||
## Product Definition
|
## Product Definition
|
||||||
|
|
||||||
A single Open WebUI **Tool** that provides LiteLLM proxy health status and cost breakdown. Designed for the Network but built for general use — any LiteLLM proxy instance.
|
A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost breakdown, and recent request history. Designed for the Network but built for general use — any LiteLLM proxy instance.
|
||||||
|
|
||||||
## OWI Integration Model
|
## OWI Integration Model
|
||||||
|
|
||||||
@@ -157,6 +157,67 @@ By Date:
|
|||||||
| Timeout (>30s) | Timeout message (large date ranges may be slow) |
|
| Timeout (>30s) | Timeout message (large date ranges may be slow) |
|
||||||
| No results | "No cost data found for [range]" |
|
| No results | "No cost data found for [range]" |
|
||||||
|
|
||||||
|
## Function 3: Request History
|
||||||
|
|
||||||
|
**LiteLLM Endpoint:** `GET {litellm_url}/spend/logs`
|
||||||
|
**Auth:** `Authorization: Bearer {litellm_master_key}`
|
||||||
|
**Params:** `limit` (int, 1-50, default 20), `show_cost` (bool, default true)
|
||||||
|
|
||||||
|
### Response Processing
|
||||||
|
|
||||||
|
The `/spend/logs` endpoint returns **all** historical request logs (no server-side pagination). The tool sorts by `startTime` descending and takes the most recent `limit` entries client-side.
|
||||||
|
|
||||||
|
**Performance note:** For large deployments (thousands of logs), this endpoint can be slow. The tool caps `limit` at 50 and uses a 30s timeout.
|
||||||
|
|
||||||
|
### Per-Request Fields
|
||||||
|
|
||||||
|
| Field | Description |
|
||||||
|
|-------|-------------|
|
||||||
|
| `request_id` | Unique request identifier |
|
||||||
|
| `model` | Model used (provider prefix stripped for readability) |
|
||||||
|
| `call_type` | Request type: `acompletion` → "chat", `aembedding` → "embed", `aimage_generation` → "image" |
|
||||||
|
| `status` | ✅ OK if no exception, ❌ FAIL if exception present |
|
||||||
|
| `startTime` | Request start timestamp |
|
||||||
|
| `request_duration_ms` | Total duration |
|
||||||
|
| `prompt_tokens` | Input tokens |
|
||||||
|
| `completion_tokens` | Output tokens |
|
||||||
|
| `cache_read_input_tokens` | Cache hit count |
|
||||||
|
| `spend` | Cost in USD |
|
||||||
|
| `metadata.exception` | Error message if failed |
|
||||||
|
|
||||||
|
### Output Format
|
||||||
|
|
||||||
|
```
|
||||||
|
LiteLLM Request History — http://host.docker.internal:4000
|
||||||
|
Showing 20 of 4560 total requests (most recent first)
|
||||||
|
==========================================================================================
|
||||||
|
1. 06-05T01:15:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 15.0s | 96→1.6k | $0.0947
|
||||||
|
2. 06-05T01:12:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 13.3s | 101→1.5k | $0.0922
|
||||||
|
3. 06-05T01:11:35 | | | ✅ OK | 0ms
|
||||||
|
4. 06-05T01:07:36 | gemini-3.5-flash | chat | ✅ OK | 8.8s | 149.8k→1.1k | $0.1821
|
||||||
|
...
|
||||||
|
|
||||||
|
Summary (20 shown): $3.1493 spent | 2,798,587 prompt + 5,815 completion tokens | 0 failed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Token Display Rules
|
||||||
|
|
||||||
|
- Prompt tokens ≥ 1000: shown as "X.Xk" (e.g., 149.8k)
|
||||||
|
- Completion tokens ≥ 1000: shown as "X.Xk"
|
||||||
|
- Cache hit % shown as 💾 indicator when `cache_read_input_tokens > 0`
|
||||||
|
- Zero-token entries (health checks, failed routing) shown without token info
|
||||||
|
|
||||||
|
### Error Paths
|
||||||
|
|
||||||
|
| Condition | Response |
|
||||||
|
|-----------|----------|
|
||||||
|
| No master key configured | Error — key required |
|
||||||
|
| 401 Unauthorized | Auth failed message |
|
||||||
|
| 403 Forbidden | Permission denied message |
|
||||||
|
| Connection refused | Connection failed message |
|
||||||
|
| Timeout (>30s) | Timeout message |
|
||||||
|
| No logs | "No request history found" |
|
||||||
|
|
||||||
## Additional Endpoints (Future Consideration)
|
## Additional Endpoints (Future Consideration)
|
||||||
|
|
||||||
| Endpoint | Purpose | Priority |
|
| Endpoint | Purpose | Priority |
|
||||||
@@ -169,7 +230,7 @@ These are documented but not in scope for v1. The two-function design is the rig
|
|||||||
|
|
||||||
## Design Principles
|
## Design Principles
|
||||||
|
|
||||||
1. **Two functions, that's it.** No scope creep. Health check and cost breakdown.
|
1. **Three functions, that's it.** No scope creep. Health check, cost breakdown, request history.
|
||||||
2. **Portable.** No hardcoded URLs or keys. Valves pass everything.
|
2. **Portable.** No hardcoded URLs or keys. Valves pass everything.
|
||||||
3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps.
|
3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps.
|
||||||
4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy.
|
4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# LiteLLM Dashboard — Open WebUI Tool
|
# LiteLLM Dashboard — Open WebUI Tool
|
||||||
|
|
||||||
A simple, two-function Open WebUI Tool for checking LiteLLM proxy health and cost breakdowns.
|
A simple, three-function Open WebUI Tool for checking LiteLLM proxy health, cost breakdowns, and recent request history.
|
||||||
|
|
||||||
## Functions
|
## Functions
|
||||||
|
|
||||||
@@ -10,6 +10,15 @@ Check LiteLLM proxy health and authentication status. Returns which models are h
|
|||||||
### `litellm_cost`
|
### `litellm_cost`
|
||||||
Get LiteLLM cost breakdown for a date range. Returns per-model and per-day spend, token counts, cache hit rate, and request stats.
|
Get LiteLLM cost breakdown for a date range. Returns per-model and per-day spend, token counts, cache hit rate, and request stats.
|
||||||
|
|
||||||
|
### `litellm_history`
|
||||||
|
Get recent LiteLLM request history. Returns the most recent N requests with model, call type, status, duration, token counts, cache hit %, and per-request cost. Useful for understanding which models are being called and how much individual requests cost.
|
||||||
|
|
||||||
|
**Parameters:**
|
||||||
|
- `limit` (int, 1-50, default 20): Number of recent requests to show
|
||||||
|
- `show_cost` (bool, default true): Include per-request cost
|
||||||
|
|
||||||
|
**Cost awareness:** This function hits `/spend/logs` which returns all historical logs. For large deployments, this can be a heavy query. Use reasonable `limit` values.
|
||||||
|
|
||||||
## Valves (Configuration)
|
## Valves (Configuration)
|
||||||
|
|
||||||
| Valve | Default | Purpose |
|
| Valve | Default | Purpose |
|
||||||
@@ -33,7 +42,7 @@ Both valves are passed at runtime — no hardcoded keys. The tool is portable to
|
|||||||
|
|
||||||
## Design Principles
|
## Design Principles
|
||||||
|
|
||||||
- **Two functions, that's it.** No scope creep.
|
- **Three functions, that's it.** No scope creep.
|
||||||
- **Portable.** No hardcoded URLs or keys.
|
- **Portable.** No hardcoded URLs or keys.
|
||||||
- **Token-efficient.** Strips `litellm_metadata` (~2KB/model) from health responses. Formats cost data as readable text, not raw JSON dumps.
|
- **Token-efficient.** Strips `litellm_metadata` (~2KB/model) from health responses. Formats cost data as readable text, not raw JSON dumps.
|
||||||
- **General-purpose.** Works with any LiteLLM proxy instance.
|
- **General-purpose.** Works with any LiteLLM proxy instance.
|
||||||
|
|||||||
Reference in new issue
Block a user