feat(docs): add VAI research and improvement plans for Hardcore FinOps v2

docs/vai/improvement_plan.md

@@ -0,0 +1,68 @@
+# Research Report: Suggested Improvements for LiteLLM Dashboard Tool
+**Target Platform:** Open WebUI (OWI)
+**Status:** Combat-Verified via Live LiteLLM Node (mon-serv1:4001)
+
+## 1. Executive Summary
+The current tool (`v1.3.2`) is a robust foundation but exhibits gaps in **Multi-Modal Spend Visibility**, **Infrastructure Health Depth**, and **Context Caching Verification**. Based on live API analysis and 2026-era LiteLLM features, the following improvements are recommended to elevate the tool to a **"System-Level Observability Hub."**
+
+---
+
+## 2. Infrastructure & Health Improvements
+
+### 2.1 Multi-Tier Readiness Probes
+**Observation:** The tool only hits `/health`. Live documentation confirms `/health/readiness` and `/health/liveliness` provide critical data on DB/Cache connectivity.
+- **Recommendation:** Add a header section to `litellm_health` that displays Proxy Readiness (DB status, Cache status, and LiteLLM Version).
+- **Endpoint:** `GET /health/readiness`
+
+### 2.2 Shared Health Check Awareness
+**Observation:** Large deployments use Redis-backed shared health checks.
+- **Recommendation:** Display the `last_updated` timestamp and shared state status. This prevents the user from assuming a "Healthy" status is fresh when it might be a cached background result from 5 minutes ago.
+
+---
+
+## 3. FinOps & Cost Breakdown Improvements
+
+### 3.1 Non-Chat Usage Visibility (Image/OCR/Audio)
+**Observation:** Live activity data (`detailed_activity.json`) shows significant spend on `/image/generations` and `/images/edits`. Currently, these are aggregated into model names, which can be confusing for multi-modal models.
+- **Recommendation:** Add an "Usage by Endpoint" section to `litellm_cost`. This would clearly delineate spend between `Chat`, `Image`, `Embed`, and `OCR`.
+- **Logic:** Parse `data['breakdown']['endpoints']`.
+
+### 3.2 Cache Creation Cost Audit
+**Observation:** The tool audits `cache_read_input_tokens` (discounts) but ignores the potential cost of **Cache Creation**. Some providers charge a premium for the first time a block is cached.
+- **Recommendation:** Update `_audit_cost` and `MODEL_PRICING` to include a `cache_creation_rate`. Add a line to the Cost Report for "Cache Inventory Growth" (Total tokens created for cache).
+
+---
+
+## 4. Request History & Audit Engine Improvements
+
+### 4.1 Enterprise Logging-v2 Support
+**Observation:** Modern LiteLLM instances (1.81.3+) support detailed `prompt_tokens_details`.
+- **Recommendation:** The current extraction logic handles `usage_object` but should be explicitly extended to support the top-level `cached_tokens` key in the `prompt_tokens_details` dictionary for 100% parity with O1/O3-class models.
+
+### 4.2 Error Tracing Depth
+**Observation:** Live logs show specific exception types (e.g., `ContextWindowExceededError`).
+- **Recommendation:** When `is_failed` is true, color-code or emoji-tag the specific failure type (e.g., 📏 for Context, 💳 for Budget, 🚦 for Rate Limit) to allow the human to diagnose routing issues at a glance.
+
+---
+
+## 5. Technical Specification for v1.4.0 (Draft)
+
+### Proposed Valve Additions
+| Valve | Default | Purpose |
+|-------|---------|---------|
+| `show_infrastructure_status` | `True` | Include DB/Redis health in `/health` output |
+| `audit_strict_mode` | `False` | Raise OWI notification if cost delta > 5% |
+
+### Proposed Data Structure Update
+```python
+# Add to MODEL_PRICING
+"gemini/gemini-3.1-flash-image-preview": {
+    "input_uncached": 1.25 / 1_000_000,
+    "input_cached": 0.125 / 1_000_000,
+    "output": 10.0 / 1_000_000,
+}
+```
+
+---
+**Report Generated:** 2026-06-27
+**Verified By:** Azure-Core (Codename: Azure Fire)