* feat(vertex_ai): support explicit AWS credentials for WIF auth
The current Vertex AI AWS Workload Identity Federation implementation
exclusively uses google.auth.aws.Credentials.from_info(), which requires
EC2 instance metadata access to obtain AWS credentials. In environments
where the metadata service is blocked for security reasons, this makes
WIF unusable.
Add support for explicit AWS credentials by implementing a custom
AwsSecurityCredentialsSupplier (google-auth >= 2.29.0). When aws_* keys
(e.g. aws_role_name, aws_region_name) are present in the WIF credential
JSON, LiteLLM uses BaseAWSLLM.get_credentials() to obtain AWS creds via
STS AssumeRole (or any other supported AWS auth flow), wraps them in the
custom supplier, and passes them to aws.Credentials() — bypassing the
metadata service entirely.
When no aws_* keys are present, the existing from_info() flow is used
unchanged, preserving full backward compatibility.
* refactor(vertex_ai): extract AWS WIF auth to own class + add docs
Address PR review feedback:
- Move _AWS_CREDENTIAL_KEYS, _extract_aws_params(), and
_credentials_from_aws_with_explicit_auth() from VertexBase into
new VertexAIAwsWifAuth class in vertex_ai_aws_wif.py
- Add documentation for explicit AWS credentials WIF auth method
in vertex.md (supported params, JSON example, SDK/Proxy tabs)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(vertex_ai): use lazy credentials provider to prevent stale STS tokens
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unused `completed_jobs` list (dead code after per-job update refactor)
- Wrap DB update in try/except to prevent one failed update from aborting remaining jobs
- Add test assertions verifying batch_processed, status, and file_object are written to DB
CheckBatchCost poller updated the status column but not the file_object
JSON column. The list_batches endpoint reads status from file_object,
so batches appeared stuck in "validating" even after Azure reported
them as completed. Now update file_object alongside status in the
per-job DB write.
Function calls not supported with reasoning_effort != 'none' on gpt-5.4.
Drop reasoning_effort when tools are in the request (small minority of volume).
Made-with: Cursor
The OpenAI chat completion API expects reasoning_effort as a string
('none', 'low', 'medium', 'high', 'xhigh'). Config/deployments may pass
the Responses API format: {'effort': 'high', 'summary': 'detailed'}.
Fix BadRequestError when model config uses dict format by extracting
the 'effort' value before passing to the API.
Made-with: Cursor
When forward_llm_provider_auth_headers=true, Authorization: Bearer <litellm-key> was
being forwarded to Anthropic if it looked like an OAuth key, causing auth failures.
Now checked against authenticated_with_header: if Authorization was used to authenticate
with the proxy, it is always stripped before forwarding to the LLM provider.
Made-with: Cursor
- Add forward_llm_provider_auth_headers support from litellm_settings
- When enabled, client x-api-key takes precedence over deployment keys
- Forward x-api-key when x-litellm-api-key or Authorization used for auth
- Fix duplicate patch lines in test_byok_oauth_endpoints.py
- Add Claude Code BYOK documentation with /login and ANTHROPIC_CUSTOM_HEADERS
- Add unit tests for clean_headers x-api-key forwarding logic
- Sync model_prices backup (pre-commit hook)
Made-with: Cursor
- Pass request_model to Azure AI cost calculator to detect router requests
- Add router flat cost ($0.14/M input tokens) even when Azure returns actual model in response
- Add test for router flat cost with response containing actual model
- Update docs with cost calculation flow and configuration requirements
Made-with: Cursor
- Shift from hardcoded model checks to dynamic lookup via _supports_factory
- Add supports_none_reasoning_effort for gpt-5.1/5.2/5.4 chat variants
- Add supports_xhigh_reasoning_effort for gpt-5.1-codex-max, gpt-5.2, gpt-5.4+
- Update model_prices_and_context_window.json and backup
- Add ProviderSpecificModelInfo types for new fields
- Fix Azure: use _supports_reasoning_effort_level instead of removed is_model_gpt_5_1_model
Made-with: Cursor
Support passing duration=null on /key/update to reset a key's expiry to never expires, alongside the existing "-1" magic string (kept for backward compat).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OpenAI gpt-5.1, gpt-5.2, and gpt-5.3 chat models all support the
`web_search_options` parameter, but the model cost registry was missing
the `supports_web_search` flag. Only `gpt-5.2-pro` had it set.
Models updated:
- gpt-5.1, gpt-5.1-2025-11-13, gpt-5.1-chat-latest
- gpt-5.2, gpt-5.2-2025-12-11, gpt-5.2-chat-latest
- gpt-5.3-chat-latest
* fix(chat): fix router.push paths to use /ui/chat with serverRootPath support
* fix(chat): wrap chat page in Suspense boundary for Next.js static export
* fix(chat): fix clipboard writeText rejection handler - remove undefined message.error call
* feat(chat): rebuild UI with routing fixes
* fix(chat): use useTheme logoUrl + /get_image fallback for sidebar logo
* feat(chat): rebuild UI with logo fix
* fix(chat): use /get_image directly for logo (no ThemeProvider outside dashboard layout)
* feat(chat): add multi-model comparison and provider logos in chat UI
- Replace single model selector with multi-select (up to 3 models)
- Show provider logos next to model names in dropdown (openai, anthropic, gemini, mistral, groq, etc.)
- Selected models float to the top of the dropdown list
- Multi-model mode: responses stream in parallel side-by-side cards below each user message
- Multi-turn: each follow-up message carries full per-model history as context
- Surface API errors inline in response cards instead of silently swallowing them
- Rebuild UI
The /organization/list endpoint only checked for PROXY_ADMIN role,
causing PROXY_ADMIN_VIEW_ONLY users to fall into the else branch
which restricts results to orgs the user is a member of. Use the
existing _user_has_admin_view() helper to include both roles.
The /team/daily/activity endpoint used Prisma pagination (page_size=1000)
but the UI only fetched page 1. Teams with many keys/models easily exceed
1000 rows in LiteLLM_DailyTeamSpend, causing truncated totals.
Switches the endpoint to use SQL GROUP BY via get_daily_activity_aggregated
with include_entity_breakdown=True, returning all data in a single response
while preserving per-team breakdown. Also adds timezone parameter support.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the messages or response JSON fields in spend logs are truncated
before being written to the database, the truncation marker now includes
a note explaining:
- This is a DB storage safeguard
- Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.)
- The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit
Also emits a verbose_proxy_logger.info message when truncation occurs in
the request body or response spend log paths.
Adds 3 new tests:
- test_truncation_includes_db_safeguard_note
- test_response_truncation_logs_info_message
- test_request_body_truncation_logs_info_message
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
After a server restart, RBAC flags (disable_agents_for_internal_users, etc.)
were only loaded into general_settings lazily when get_ui_settings or
update_ui_settings was called. This meant restrictions were silently bypassed
until someone hit one of those endpoints.
Add _sync_ui_settings_to_general_settings() to ProxyStartupEvent that reads
the litellm_uisettings DB record and syncs _RUNTIME_GENERAL_SETTINGS_FLAGS
into general_settings immediately after prisma_client is initialized.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, model_dump(exclude_none=True) included all bool fields (since
False != None), causing a partial PATCH to overwrite every other setting to
its default. Fix uses exclude_unset=True and reads the existing DB record
before merging, giving proper PATCH semantics.
This was a pre-existing bug but is fixed here since we're touching this code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- rbac_utils.py: change feature_name from str to Literal["agents", "vector_stores"]
so typos are caught by type checkers at import time
- proxy_setting_endpoints.py: extract _RUNTIME_GENERAL_SETTINGS_FLAGS as a module-level
constant, replacing duplicated inline lists in get_ui_settings and update_ui_settings
- test_vector_store_rbac.py: remove try/except pattern that silently swallowed non-403
HTTPExceptions; tests now let any unexpected exception propagate as a test failure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- rbac_utils.py: remove duplicated _check_if_team_admin/_is_user_team_admin_for_any_team;
delegate to _user_has_admin_privileges from management_endpoints/common_utils with the
shared user_api_key_cache (fixes no-op DualCache and missing org admin coverage)
- test_rbac_utils.py: update patch target to match new delegation path
- SidebarProvider.tsx: pass allowAgentsForTeamAdmins and allowVectorStoresForTeamAdmins
props to Sidebar
- leftnav.tsx: add useTeams hook + isTeamAdmin memo; exempt team admins from sidebar
filtering when allow_*_for_team_admins is enabled (fixes frontend/backend inconsistency)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 58 unit tests across 5 files in ui/litellm-dashboard/src/components/policies/:
- build_attachment_data.test.ts: pure logic tests for global vs specific scope
- impact_preview_alert.test.tsx: rendering for global scope warning and specific scope counts/tags
- PolicySelector.test.tsx: pure function tests for getPolicyOptionEntries/policyVersionRef + component fetch/disable behavior
- policy_info.test.tsx: loading, not-found, policy details rendering, admin gating, onClose/onEdit callbacks
- attachment_table.test.tsx: loading/empty states, row rendering, scope badges, delete admin gating
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude Code v2.1.69+ sends `custom: {defer_loading: true}` on tool
definitions. Anthropic's API accepts this field, but Bedrock rejects it
with "Extra inputs are not permitted", causing ~90% of requests to fail.
Strip the `custom` field from each tool in the request body before
sending to Bedrock, in both the Messages API and Chat API invoke paths.
Fixes#22847
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
