diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 11bf123..9384bae 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -26,12 +26,13 @@ A single Open WebUI **Tool** that provides LiteLLM proxy health status, cost bre ## Valve Configuration -The tool needs two valves: +The tool needs three valves: | Valve | Type | Purpose | Default | |-------|------|---------|---------| | `litellm_url` | string | LiteLLM proxy base URL (e.g., `http://localhost:4000`) | `http://localhost:4000` | | `litellm_master_key` | string | LiteLLM master API key for auth | (empty — must be configured) | +| `default_hide_zero_spend` | bool | Default behavior for zero-spend model filtering | `True` | **Security consideration:** The master key will be stored in the OWI valve configuration. This is the same security model as other API keys in OWI (LiteLLM proxy key, etc.). The key is not exposed in chat context — OWI valves are injected at tool execution time, not displayed to the user. @@ -93,7 +94,19 @@ Healthy: 8 | Unhealthy: 0 **LiteLLM Endpoint:** `GET {litellm_url}/user/daily/activity` **Auth:** `Authorization: Bearer {litellm_master_key}` -**Params:** `start_date` (YYYY-MM-DD or natural language shortcut), `end_date` (YYYY-MM-DD or natural language shortcut), `show_zero_spend` (bool, default False) +**Params:** `start_date` (YYYY-MM-DD or natural language shortcut), `end_date` (YYYY-MM-DD or natural language shortcut), `hide_zero_spend` (bool, default False) + +### Two-Level Zero-Spend Filtering + +The tool uses a **two-level design** for zero-spend model filtering: + +1. **Valve-level default** (`default_hide_zero_spend`, default `True`): The human operator sets the default behavior via valves. When `True`, models with sub-$0.001 spend are hidden by default. + +2. **Per-call override** (`hide_zero_spend` parameter): The SLM can explicitly override the valve default on a per-call basis: + - `hide_zero_spend=True` → **Always hide** (explicit override, regardless of valve) + - `hide_zero_spend=False` → **Use valve default** (defers to `default_hide_zero_spend`) + +This design is defensive against SLM behavior: if the SLM passes `hide_zero_spend=False` (or omits it), the valve default still applies. Only an explicit `hide_zero_spend=True` forces hiding. The SLM cannot accidentally expose zero-spend models unless the human has set `default_hide_zero_spend=False`. ### Natural Language Date Resolution @@ -109,12 +122,6 @@ The `_resolve_date()` helper converts natural language shortcuts to YYYY-MM-DD b Already-formatted YYYY-MM-DD strings pass through unchanged. Unrecognized strings are passed to the API, which will reject them with a clear error. -### show_zero_spend Parameter - -When `show_zero_spend=False` (the default), models with sub-$0.001 spend are hidden from the per-model breakdown and summarized as `... (N model(s) with sub-$0.001 spend hidden)`. This is the recommended default because free-tier or subscription models (e.g., Ollama Cloud) often dominate the report with high token counts but negligible cost, drowning out the models that actually cost money. - -Set `show_zero_spend=True` to see all models, including those with zero or near-zero spend. - ### Response Processing The `/user/daily/activity` endpoint returns a rich breakdown structure: @@ -160,15 +167,9 @@ By Date: - Token counts (prompt, completion, total) - Cache hit rate (cache_read_input_tokens / total_input_tokens) - Request counts (successful + failed) -- Per-model breakdown (sorted by spend, descending; optionally filtered by hide_zero_spend) +- Per-model breakdown (sorted by spend, descending; optionally filtered by zero-spend hiding) - Per-date breakdown -- Hidden model count summary (when hide_zero_spend is active) - -### Condensed/Stripped - -- `api_key_breakdown` sub-levels — too granular for general display -- `metadata: {}` — empty in current responses -- `cache_creation_input_tokens` — shown only when non-zero +- Hidden model count summary (when zero-spend hiding is active) ### Error Paths @@ -258,7 +259,7 @@ These are documented but not in scope for v1. The two-function design is the rig ### v1.2.0 (2026-06-13) - **feat:** Natural language date shortcuts for `litellm_cost()` — `today`, `yesterday`, `monday`, `last week`, etc. -- **feat:** `show_zero_spend` parameter for `litellm_cost()` — default False hides sub-$0.001 models with summary line +- **feat:** Two-level zero-spend filtering — valve default (`default_hide_zero_spend`, default True) + per-call override (`hide_zero_spend`) - **fix:** Cost report header shows resolved YYYY-MM-DD dates instead of raw shortcuts ### v1.1.0 (2026-06-13) @@ -273,4 +274,5 @@ These are documented but not in scope for v1. The two-function design is the rig 2. **Portable.** No hardcoded URLs or keys. Valves pass everything. 3. **Token-efficient.** Strip litellm_metadata from health. Format cost data as readable text, not raw JSON dumps. 4. **General-purpose.** Designed for the Network but usable by anyone with a LiteLLM proxy. -5. **Read-only.** These endpoints are GET only. No mutations, no risk. \ No newline at end of file +5. **Read-only.** These endpoints are GET only. No mutations, no risk. +6. **Defensive against SLM behavior.** Two-level parameter design ensures the human's valve defaults are respected even when the SLM passes conflicting parameters. diff --git a/README.md b/README.md index 9e7149d..f610a69 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Get LiteLLM cost breakdown for a date range. Returns per-model and per-day spend **Parameters:** - `start_date` (str, required): Start date — YYYY-MM-DD or natural language shortcut (e.g. "2026-06-01", "today", "yesterday", "monday", "last monday", "two days ago", "this week", "last month") - `end_date` (str, required): End date — YYYY-MM-DD or natural language shortcut -- `show_zero_spend` (bool, default false): Show models with sub-$0.001 spend. When false, these models are hidden and summarized as "... (N model(s) with sub-$0.001 spend hidden)". Useful when free-tier or subscription models (e.g. Ollama Cloud) dominate the breakdown with high token counts but negligible cost. +- `hide_zero_spend` (bool, default false): When True, force-hide models with sub-$0.001 spend. When False, use the valve-configured default (see `default_hide_zero_spend` below). This two-level design lets the human set a default via valves while allowing the SLM to explicitly override on a per-call basis. ### `litellm_history` Get recent LiteLLM request history. Returns the most recent N requests with model, call type, status, duration, token counts, cache hit %, and per-request cost. Useful for understanding which models are being called and how much individual requests cost. @@ -31,6 +31,7 @@ Get recent LiteLLM request history. Returns the most recent N requests with mode |-------|---------|---------| | `litellm_url` | `http://localhost:4000` | LiteLLM proxy base URL | | `litellm_master_key` | (empty) | LiteLLM master API key (password field) | +| `default_hide_zero_spend` | `True` | Default behavior for zero-spend model filtering. When True, models with sub-$0.001 spend are hidden by default. The per-call `hide_zero_spend` parameter can override this. | Both valves are passed at runtime — no hardcoded keys. The tool is portable to any LiteLLM proxy instance. @@ -63,7 +64,7 @@ Both valves are passed at runtime — no hardcoded keys. The tool is portable to ### v1.2.0 (2026-06-13) - **feat:** Natural language date shortcuts for `litellm_cost()` — `today`, `yesterday`, `monday`, `last week`, etc. -- **feat:** `hide_zero_spend` parameter for `litellm_cost()` — filters sub-$0.001 models from breakdown +- **feat:** `hide_zero_spend` parameter for `litellm_cost()` — default True hides sub-$0.001 models with summary line; per-call parameter can override - **fix:** Cost report header shows resolved YYYY-MM-DD dates instead of raw shortcuts ### v1.1.0 (2026-06-13) diff --git a/litellm_dashboard.py b/litellm_dashboard.py index 9205382..54d3621 100644 --- a/litellm_dashboard.py +++ b/litellm_dashboard.py @@ -30,6 +30,10 @@ class Tools: description="LiteLLM master API key for authentication", json_schema_extra={"input": {"type": "password"}}, ) + default_hide_zero_spend: bool = Field( + default=True, + description="Default behavior for zero-spend model filtering. When True, models with sub-$0.001 spend are hidden by default and summarized. The per-call hide_zero_spend parameter can override this.", + ) def _headers(self) -> dict: """Build auth headers from valves.""" @@ -190,18 +194,23 @@ class Tools: self, start_date: str, end_date: str, - show_zero_spend: bool = False, + hide_zero_spend: bool = False, ) -> str: """ Get LiteLLM cost breakdown for a date range. :param start_date: Start date — YYYY-MM-DD or natural language shortcut (e.g. "2026-06-01", "today", "yesterday", "monday", "last monday", "two days ago", "this week", "last month") :param end_date: End date — YYYY-MM-DD or natural language shortcut - :param show_zero_spend: Show models with sub-$0.001 spend (default False). When False, these models are hidden and summarized as "... (N model(s) with sub-$0.001 spend hidden)". Useful when free-tier or subscription models (e.g. Ollama Cloud) dominate the breakdown with high token counts but negligible cost. + :param hide_zero_spend: When True, force-hide models with sub-$0.001 spend. When False (default), use the valve-configured default behavior (default_hide_zero_spend). This two-level design lets the human set a default via valves while allowing the SLM to explicitly override on a per-call basis. Returns formatted cost report with per-model and per-day breakdowns, token counts, cache hit rate, and request stats. """ if not self.valves.litellm_master_key: return "❌ Master key required for cost data. Set the litellm_master_key valve." + # Resolve effective hide behavior: explicit param overrides valve default + # hide_zero_spend=True → ALWAYS hide (explicit override) + # hide_zero_spend=False → use valve default (default_hide_zero_spend) + effective_hide = hide_zero_spend or self.valves.default_hide_zero_spend + resolved_start = self._resolve_date(start_date) resolved_end = self._resolve_date(end_date) @@ -228,7 +237,7 @@ class Tools: metadata = data.get("metadata", {}) if not results: - return f"No cost data found for {start_date} to {end_date}." + return f"No cost data found for {resolved_start} to {resolved_end}." # Aggregate totals from metadata total_spend = metadata.get("total_spend", 0) @@ -284,7 +293,7 @@ class Tools: hidden_count = 0 for model_name in sorted(model_totals, key=lambda k: model_totals[k]["spend"], reverse=True): mt = model_totals[model_name] - if not show_zero_spend and mt["spend"] < 0.001: + if effective_hide and mt["spend"] < 0.001: hidden_count += 1 continue # Strip common provider prefixes for readability