feat: two-level zero-spend filtering with valve default
Adds default_hide_zero_spend valve (default True) that controls whether models with sub-bash.001 spend are hidden by default. The per-call hide_zero_spend parameter can explicitly override: - hide_zero_spend=True → ALWAYS hide (explicit override) - hide_zero_spend=False → use valve default (default_hide_zero_spend) This defensive design prevents SLMs from accidentally exposing zero-spend models unless the human has explicitly set default_hide_zero_spend=False. - Added default_hide_zero_spend valve to Valves class - Updated litellm_cost() with two-level logic - Updated README and ARCHITECTURE.md with two-level design docs - Reverted parameter name back to hide_zero_spend
This commit is contained in:
1 parent
091d44ab87
commit
097a29eaee
3 files changed
+36
-24
No files matched your search
+20
-18
@@ -26,12 +26,13 @@ A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost bre
|
||||
|
||||
## Valve Configuration
|
||||
|
||||
The tool needs two valves:
|
||||
The tool needs three valves:
|
||||
|
||||
| Valve | Type | Purpose | Default |
|
||||
|-------|------|---------|---------|
|
||||
| `litellm_url` | string | LiteLLM proxy base URL (e.g., `http://localhost:4000`) | `http://localhost:4000` |
|
||||
| `litellm_master_key` | string | LiteLLM master API key for auth | (empty — must be configured) |
|
||||
| `default_hide_zero_spend` | bool | Default behavior for zero-spend model filtering | `True` |
|
||||
|
||||
**Security consideration:** The master key will be stored in the OWI valve configuration. This is the same security model as other API keys in OWI (LiteLLM proxy key, etc.). The key is not exposed in chat context — OWI valves are injected at tool execution time, not displayed to the user.
|
||||
|
||||
@@ -93,7 +94,19 @@ Healthy: 8 | Unhealthy: 0
|
||||
|
||||
**LiteLLM Endpoint:** `GET {litellm_url}/user/daily/activity`
|
||||
**Auth:** `Authorization: Bearer {litellm_master_key}`
|
||||
**Params:** `start_date` (YYYY-MM-DD or natural language shortcut), `end_date` (YYYY-MM-DD or natural language shortcut), `show_zero_spend` (bool, default False)
|
||||
**Params:** `start_date` (YYYY-MM-DD or natural language shortcut), `end_date` (YYYY-MM-DD or natural language shortcut), `hide_zero_spend` (bool, default False)
|
||||
|
||||
### Two-Level Zero-Spend Filtering
|
||||
|
||||
The tool uses a **two-level design** for zero-spend model filtering:
|
||||
|
||||
1. **Valve-level default** (`default_hide_zero_spend`, default `True`): The human operator sets the default behavior via valves. When `True`, models with sub-$0.001 spend are hidden by default.
|
||||
|
||||
2. **Per-call override** (`hide_zero_spend` parameter): The SLM can explicitly override the valve default on a per-call basis:
|
||||
- `hide_zero_spend=True` → **Always hide** (explicit override, regardless of valve)
|
||||
- `hide_zero_spend=False` → **Use valve default** (defers to `default_hide_zero_spend`)
|
||||
|
||||
This design is defensive against SLM behavior: if the SLM passes `hide_zero_spend=False` (or omits it), the valve default still applies. Only an explicit `hide_zero_spend=True` forces hiding. The SLM cannot accidentally expose zero-spend models unless the human has set `default_hide_zero_spend=False`.
|
||||
|
||||
### Natural Language Date Resolution
|
||||
|
||||
@@ -109,12 +122,6 @@ The `_resolve_date()` helper converts natural language shortcuts to YYYY-MM-DD b
|
||||
|
||||
Already-formatted YYYY-MM-DD strings pass through unchanged. Unrecognized strings are passed to the API, which will reject them with a clear error.
|
||||
|
||||
### show_zero_spend Parameter
|
||||
|
||||
When `show_zero_spend=False` (the default), models with sub-$0.001 spend are hidden from the per-model breakdown and summarized as `... (N model(s) with sub-$0.001 spend hidden)`. This is the recommended default because free-tier or subscription models (e.g., Ollama Cloud) often dominate the report with high token counts but negligible cost, drowning out the models that actually cost money.
|
||||
|
||||
Set `show_zero_spend=True` to see all models, including those with zero or near-zero spend.
|
||||
|
||||
### Response Processing
|
||||
|
||||
The `/user/daily/activity` endpoint returns a rich breakdown structure:
|
||||
@@ -160,15 +167,9 @@ By Date:
|
||||
- Token counts (prompt, completion, total)
|
||||
- Cache hit rate (cache_read_input_tokens / total_input_tokens)
|
||||
- Request counts (successful + failed)
|
||||
- Per-model breakdown (sorted by spend, descending; optionally filtered by hide_zero_spend)
|
||||
- Per-model breakdown (sorted by spend, descending; optionally filtered by zero-spend hiding)
|
||||
- Per-date breakdown
|
||||
- Hidden model count summary (when hide_zero_spend is active)
|
||||
|
||||
### Condensed/Stripped
|
||||
|
||||
- `api_key_breakdown` sub-levels — too granular for general display
|
||||
- `metadata: {}` — empty in current responses
|
||||
- `cache_creation_input_tokens` — shown only when non-zero
|
||||
- Hidden model count summary (when zero-spend hiding is active)
|
||||
|
||||
### Error Paths
|
||||
|
||||
@@ -258,7 +259,7 @@ These are documented but not in scope for v1. The two-function design is the rig
|
||||
|
||||
### v1.2.0 (2026-06-13)
|
||||
- **feat:** Natural language date shortcuts for `litellm_cost()` — `today`, `yesterday`, `monday`, `last week`, etc.
|
||||
- **feat:** `show_zero_spend` parameter for `litellm_cost()` — default False hides sub-$0.001 models with summary line
|
||||
- **feat:** Two-level zero-spend filtering — valve default (`default_hide_zero_spend`, default True) + per-call override (`hide_zero_spend`)
|
||||
- **fix:** Cost report header shows resolved YYYY-MM-DD dates instead of raw shortcuts
|
||||
|
||||
### v1.1.0 (2026-06-13)
|
||||
@@ -273,4 +274,5 @@ These are documented but not in scope for v1. The two-function design is the rig
|
||||
2. **Portable.** No hardcoded URLs or keys. Valves pass everything.
|
||||
3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps.
|
||||
4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy.
|
||||
5. **Read-only.** These endpoints are GET only. No mutations, no risk.
|
||||
5. **Read-only.** These endpoints are GET only. No mutations, no risk.
|
||||
6. **Defensive against SLM behavior.** Two-level parameter design ensures the human's valve defaults are respected even when the SLM passes conflicting parameters.
|
||||
Reference in new issue
Block a user