docs: update README and ARCHITECTURE for v1.1.0 — litellm_history function
- README: three functions, litellm_history params and cost awareness note - ARCHITECTURE: full Function 3 section with endpoint, fields, output format, token display rules, error paths, and performance considerations
This commit is contained in:
1 parent
fd449079be
commit
8d90c0c277
2 files changed
+74
-4
No files matched your search
+63
-2
@@ -5,7 +5,7 @@
|
||||
|
||||
## Product Definition
|
||||
|
||||
A single Open WebUI **Tool** that provides LiteLLM proxy health status and cost breakdown. Designed for the Network but built for general use — any LiteLLM proxy instance.
|
||||
A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost breakdown, and recent request history. Designed for the Network but built for general use — any LiteLLM proxy instance.
|
||||
|
||||
## OWI Integration Model
|
||||
|
||||
@@ -157,6 +157,67 @@ By Date:
|
||||
| Timeout (>30s) | Timeout message (large date ranges may be slow) |
|
||||
| No results | "No cost data found for [range]" |
|
||||
|
||||
## Function 3: Request History
|
||||
|
||||
**LiteLLM Endpoint:** `GET {litellm_url}/spend/logs`
|
||||
**Auth:** `Authorization: Bearer {litellm_master_key}`
|
||||
**Params:** `limit` (int, 1-50, default 20), `show_cost` (bool, default true)
|
||||
|
||||
### Response Processing
|
||||
|
||||
The `/spend/logs` endpoint returns **all** historical request logs (no server-side pagination). The tool sorts by `startTime` descending and takes the most recent `limit` entries client-side.
|
||||
|
||||
**Performance note:** For large deployments (thousands of logs), this endpoint can be slow. The tool caps `limit` at 50 and uses a 30s timeout.
|
||||
|
||||
### Per-Request Fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `request_id` | Unique request identifier |
|
||||
| `model` | Model used (provider prefix stripped for readability) |
|
||||
| `call_type` | Request type: `acompletion` → "chat", `aembedding` → "embed", `aimage_generation` → "image" |
|
||||
| `status` | ✅ OK if no exception, ❌ FAIL if exception present |
|
||||
| `startTime` | Request start timestamp |
|
||||
| `request_duration_ms` | Total duration |
|
||||
| `prompt_tokens` | Input tokens |
|
||||
| `completion_tokens` | Output tokens |
|
||||
| `cache_read_input_tokens` | Cache hit count |
|
||||
| `spend` | Cost in USD |
|
||||
| `metadata.exception` | Error message if failed |
|
||||
|
||||
### Output Format
|
||||
|
||||
```
|
||||
LiteLLM Request History — http://host.docker.internal:4000
|
||||
Showing 20 of 4560 total requests (most recent first)
|
||||
==========================================================================================
|
||||
1. 06-05T01:15:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 15.0s | 96→1.6k | $0.0947
|
||||
2. 06-05T01:12:27 | gemini-3.1-flash-image-previ | image | ✅ OK | 13.3s | 101→1.5k | $0.0922
|
||||
3. 06-05T01:11:35 | | | ✅ OK | 0ms
|
||||
4. 06-05T01:07:36 | gemini-3.5-flash | chat | ✅ OK | 8.8s | 149.8k→1.1k | $0.1821
|
||||
...
|
||||
|
||||
Summary (20 shown): $3.1493 spent | 2,798,587 prompt + 5,815 completion tokens | 0 failed
|
||||
```
|
||||
|
||||
### Token Display Rules
|
||||
|
||||
- Prompt tokens ≥ 1000: shown as "X.Xk" (e.g., 149.8k)
|
||||
- Completion tokens ≥ 1000: shown as "X.Xk"
|
||||
- Cache hit % shown as 💾 indicator when `cache_read_input_tokens > 0`
|
||||
- Zero-token entries (health checks, failed routing) shown without token info
|
||||
|
||||
### Error Paths
|
||||
|
||||
| Condition | Response |
|
||||
|-----------|----------|
|
||||
| No master key configured | Error — key required |
|
||||
| 401 Unauthorized | Auth failed message |
|
||||
| 403 Forbidden | Permission denied message |
|
||||
| Connection refused | Connection failed message |
|
||||
| Timeout (>30s) | Timeout message |
|
||||
| No logs | "No request history found" |
|
||||
|
||||
## Additional Endpoints (Future Consideration)
|
||||
|
||||
| Endpoint | Purpose | Priority |
|
||||
@@ -169,7 +230,7 @@ These are documented but not in scope for v1. The two-function design is the rig
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Two functions, that's it.** No scope creep. Health check and cost breakdown.
|
||||
1. **Three functions, that's it.** No scope creep. Health check, cost breakdown, request history.
|
||||
2. **Portable.** No hardcoded URLs or keys. Valves pass everything.
|
||||
3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps.
|
||||
4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy.
|
||||
|
||||
Reference in new issue
Block a user