* fix: don't close HTTP/SDK clients on LLMClientCache eviction
Removing the _remove_key override that eagerly called aclose()/close()
on evicted clients. Evicted clients may still be held by in-flight
streaming requests; closing them causes:
RuntimeError: Cannot send a request, as the client has been closed.
This is a regression from commit fb72979432. Clients that are no longer
referenced will be garbage-collected naturally. Explicit shutdown cleanup
happens via close_litellm_async_clients().
Fixes production crashes after the 1-hour cache TTL expires.
* test: update LLMClientCache unit tests for no-close-on-eviction behavior
Flip the assertions: evicted clients must NOT be closed. Replace
test_remove_key_closes_async_client → test_remove_key_does_not_close_async_client
and equivalents for sync/eviction paths.
Add test_remove_key_removes_plain_values for non-client cache entries.
Remove test_background_tasks_cleaned_up_after_completion (no more _background_tasks).
Remove test_remove_key_no_event_loop variant that depended on old behavior.
* test: add e2e tests for OpenAI SDK client surviving cache eviction
Add two new e2e tests using real AsyncOpenAI clients:
- test_evicted_openai_sdk_client_stays_usable: verifies size-based eviction
doesn't close the client
- test_ttl_expired_openai_sdk_client_stays_usable: verifies TTL expiry
eviction doesn't close the client
Both tests sleep after eviction so any create_task()-based close would
have time to run, making the regression detectable.
Also expand the module docstring to explain why the sleep is required.
* docs(AGENTS.md): add rule — never close HTTP/SDK clients on cache eviction
* docs(CLAUDE.md): add HTTP client cache safety guideline
* feat(ui): add missing provider logos and map all backend providers to UI
- Downloaded 26 SVG logos from lobehub/lobe-icons for providers that were
missing visual branding (AI21, Baseten, Cloudflare, GitHub, Huggingface,
Hyperbolic, Lambda, LM Studio, Meta Llama, Moonshot, Nebius, Novita,
Nvidia NIM, Replicate, Recraft, Topaz, V0, Vercel, Watsonx/IBM,
Xinference, Friendli, Morph, Cometapi, Featherless, Langfuse, GitHub Copilot)
- Extended Providers enum from 47 to 107 entries to cover all backend
providers from provider_create_fields.json
- Extended provider_map to map all new enum keys to litellm_provider values
- Extended providerLogoMap to assign logos to all providers where available,
reusing parent logos for variants (e.g. Anthropic Text -> anthropic.svg)
- Fixed SVG currentColor issue: replaced fill='currentColor' with explicit
colors since CSS inheritance doesn't work in <img> elements
- Updated test reference from Providers.Watsonx to Providers.WATSONX
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs(agents): add UI dashboard dev notes to Cursor Cloud instructions
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* refactor(ui): remove non-LLM providers from Add Model dropdown
Remove Custom, Custom OpenAI, GitHub, Humanloop, Langfuse, Litellm Proxy,
and Milvus from the Providers enum, provider_map, and providerLogoMap.
These are not LLM API providers (they are internal tools, vector stores,
or observability platforms) and should not appear in the Add Model form.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Backend - Spend Log Storage for Realtime Calls:
- Collect user voice transcripts and text input during WebSocket sessions
- Store collected messages in spend logs when store_prompts_in_spend_logs enabled
- Capture tool definitions from session.update and tool calls from response.done
- Enrich proxy_server_request with tools and response with tool_calls for UI
Backend - WebSocket Auth:
- Support browser-based auth via Sec-WebSocket-Protocol subprotocol
- Echo back subprotocol on WebSocket accept
UI - Realtime Playground:
- New RealtimePlayground component with WebSocket voice+text chat
- Mic recording (PCM16 24kHz), server VAD, audio playback, text input
- Handle binary WebSocket frames (Blob/ArrayBuffer decoding)
- Add /v1/realtime endpoint option to playground endpoint selector
UI - Tools Section for Realtime Logs:
- Extract tool calls from realtime response format (response.tool_calls
and response.results[].response.output[].type=function_call)
Tests:
- 15 new backend tests for realtime streaming and spend log storage
- 4 new UI tests for realtime tool call extraction
Fixes pre-existing build errors:
- ToolPolicies.tsx: duplicate import, antd styles type
- create_key_button.tsx: missing message import
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): add user filtering to usage page
Adds "User Usage" as a new view option in the usage page dropdown,
allowing admins to view and filter usage data by individual users
via the existing /user/daily/activity backend endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(ui/): working usage filtering
* fix(ui): use single-select for user filter and add tests
The user entity type's backend endpoint only accepts a single user_id,
so the filter now uses single-select mode instead of multi-select.
Added tests for the new user entity type in EntityUsage and
UsageViewSelect. Updated CLAUDE.md and AGENTS.md with guidance on
UI/backend contract consistency and test coverage for new entity types.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: remove unintended package-lock.json changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: restore package-lock.json to merge base state
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove development setup files from repository
Removes VERTEX_ENV_SETUP.md and setup_vertex_env.sh from the
repository as they are not referenced in documentation or tests.
These files were added in PR #15824 alongside the VertexAI Search
feature. The setup information is already well-documented in the
official docs at:
docs/my-website/docs/pass_through/vertex_ai_search_datastores.md
* docs: add poetry run python usage to CLAUDE.md and AGENTS.md
