feat: two-level zero-spend filtering with valve default
Adds default_hide_zero_spend valve (default True) that controls whether models with sub-bash.001 spend are hidden by default. The per-call hide_zero_spend parameter can explicitly override: - hide_zero_spend=True → ALWAYS hide (explicit override) - hide_zero_spend=False → use valve default (default_hide_zero_spend) This defensive design prevents SLMs from accidentally exposing zero-spend models unless the human has explicitly set default_hide_zero_spend=False. - Added default_hide_zero_spend valve to Valves class - Updated litellm_cost() with two-level logic - Updated README and ARCHITECTURE.md with two-level design docs - Reverted parameter name back to hide_zero_spend
This commit is contained in:
1 parent
091d44ab87
commit
097a29eaee
3 files changed
+36
-24
No files matched your search
+13
-4
@@ -30,6 +30,10 @@ class Tools:
|
||||
description="LiteLLM master API key for authentication",
|
||||
json_schema_extra={"input": {"type": "password"}},
|
||||
)
|
||||
default_hide_zero_spend: bool = Field(
|
||||
default=True,
|
||||
description="Default behavior for zero-spend model filtering. When True, models with sub-$0.001 spend are hidden by default and summarized. The per-call hide_zero_spend parameter can override this.",
|
||||
)
|
||||
|
||||
def _headers(self) -> dict:
|
||||
"""Build auth headers from valves."""
|
||||
@@ -190,18 +194,23 @@ class Tools:
|
||||
self,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
show_zero_spend: bool = False,
|
||||
hide_zero_spend: bool = False,
|
||||
) -> str:
|
||||
"""
|
||||
Get LiteLLM cost breakdown for a date range.
|
||||
:param start_date: Start date — YYYY-MM-DD or natural language shortcut (e.g. "2026-06-01", "today", "yesterday", "monday", "last monday", "two days ago", "this week", "last month")
|
||||
:param end_date: End date — YYYY-MM-DD or natural language shortcut
|
||||
:param show_zero_spend: Show models with sub-$0.001 spend (default False). When False, these models are hidden and summarized as "... (N model(s) with sub-$0.001 spend hidden)". Useful when free-tier or subscription models (e.g. Ollama Cloud) dominate the breakdown with high token counts but negligible cost.
|
||||
:param hide_zero_spend: When True, force-hide models with sub-$0.001 spend. When False (default), use the valve-configured default behavior (default_hide_zero_spend). This two-level design lets the human set a default via valves while allowing the SLM to explicitly override on a per-call basis.
|
||||
Returns formatted cost report with per-model and per-day breakdowns, token counts, cache hit rate, and request stats.
|
||||
"""
|
||||
if not self.valves.litellm_master_key:
|
||||
return "❌ Master key required for cost data. Set the litellm_master_key valve."
|
||||
|
||||
# Resolve effective hide behavior: explicit param overrides valve default
|
||||
# hide_zero_spend=True → ALWAYS hide (explicit override)
|
||||
# hide_zero_spend=False → use valve default (default_hide_zero_spend)
|
||||
effective_hide = hide_zero_spend or self.valves.default_hide_zero_spend
|
||||
|
||||
resolved_start = self._resolve_date(start_date)
|
||||
resolved_end = self._resolve_date(end_date)
|
||||
|
||||
@@ -228,7 +237,7 @@ class Tools:
|
||||
metadata = data.get("metadata", {})
|
||||
|
||||
if not results:
|
||||
return f"No cost data found for {start_date} to {end_date}."
|
||||
return f"No cost data found for {resolved_start} to {resolved_end}."
|
||||
|
||||
# Aggregate totals from metadata
|
||||
total_spend = metadata.get("total_spend", 0)
|
||||
@@ -284,7 +293,7 @@ class Tools:
|
||||
hidden_count = 0
|
||||
for model_name in sorted(model_totals, key=lambda k: model_totals[k]["spend"], reverse=True):
|
||||
mt = model_totals[model_name]
|
||||
if not show_zero_spend and mt["spend"] < 0.001:
|
||||
if effective_hide and mt["spend"] < 0.001:
|
||||
hidden_count += 1
|
||||
continue
|
||||
# Strip common provider prefixes for readability
|
||||
|
||||
Reference in new issue
Block a user