FinOps Research Report: Accuracy, Efficiency, and Readability (v1.4.0)

Subject: Hardcore Observability Improvements for LiteLLM Dashboard Resource: Verified via mon-serv1.captcouch.xyz:4001 (LiteLLM 1.81.3+)

1.1 Dynamic Price Discovery

Observation: Hardcoded MODEL_PRICING in the tool is prone to drift. Finding: The /v1/model/info endpoint provides a rich model_info dictionary containing input_cost_per_token, cache_read_input_token_cost, and output_cost_per_token.

Recommendation: Implement a Hybrid Audit Engine. Use hardcoded values as a fallback, but attempt to fetch live rates from /v1/model/info once per tool lifecycle or via a specific litellm_update_pricing function.
Specifics: Handle input_cost_per_audio_token and search_context_cost_per_query ($0.014 on Gemini) to ensure audit parity for audio/web-search tasks.

1.2 Reasoning Token Awareness

Observation: OpenAI o-series and Gemini models now separate "thinking" tokens. Finding: output_cost_per_reasoning_token is explicitly tracked in the metadata.

Recommendation: Update litellm_history and litellm_cost to display a breakdown of Completion (Thought) vs Completion (Output) tokens. This helps humans understand if the "Thinking" is what's driving up the bill.

2. Token Efficiency: Summary Endpoints & Compression

2.1 Low-Bandwidth Summary

Observation: /user/daily/activity is verbose (~3000 lines for 3 days). Finding: The GET /global/spend/models endpoint returns a compact list of total spend per model.

Recommendation: Introduce a litellm_summary function. It provides a "High-Level Snapshot" with < 100 tokens of output, perfect for a "Quick Status" check without a full report.
Endpoint: GET /global/spend/models

2.2 Metadata Compression

Observation: The /health endpoint response is 90% redundant auth context. Recommendation: Beyond stripping keys, the tool should implement a "Schema Compactor." Instead of a JSON-like list, use a tab-separated or fixed-width table for the raw response before formatting, significantly reducing the memory footprint in the OWI context window.

3. Readability: Visual Hierarchies

3.1 Time-Series Visualization (Sparklines)

Recommendation: For litellm_cost, instead of just listing dates, use Unicode block characters (e.g., ▂ ▃ ▅ ▆ ▇) to show the spend trend over the requested period.

Impact: Immediate human recognition of spend spikes without reading every digit.

3.2 Tiered Alerts

Recommendation: Implement color-coded status flags in the OWI markdown output:

[CRITICAL] Spend > 90% of a key's max_budget (found in GET /spend/keys).
[AUDIT_FAIL] Cost delta > 10% in the audit engine.
[STALE] Health status > 10 minutes old.

4. Proposed Implementation Path

Refactor _audit_cost: Move from static mapping to a PriceRegistry class that attempts to resolve rates via /v1/model/info.
Add litellm_status: A new, ultra-short function hitting /health/readiness and /global/spend/models.
Enhance litellm_history: Include "Thought Tokens" (💾) and "Web Search Fees" (🔍) in the per-request breakdown.

Verified: 2026-06-27 | Azure-Core System: High-Performance Observability Node

3.3 KiB Raw Blame History