PR #22850 (BYOK MCP servers) accidentally re-declared spec_path which was
already added by PR #22820, causing Prisma schema validation to fail with
error P1012 "Field is already defined".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- batches/main.py: import FileExpiresAfter, cast output_expires_after on assignment
- openai/openai.py, azure/batches/handler.py: add # type: ignore[arg-type] on
batches.create / batches.retrieve TypedDict unpacking calls
- searchapi/transformation.py: cast optional_params["country"] to str before .lower()
- openrouter/image_edit/transformation.py: cast iterated value to str for size/quality params
- spend_log_cleanup.py: narrow bool | None to bool with `or False`
- cost_tracking_settings.py: cast base_model/resolved_model to str and
custom_llm_provider to Optional[str] in return statements
- text_moderation.py: suppress misc TypedDict ** expansion error; use cast for response
- prompt_shield.py: use cast instead of TypedDict(**response_json) construction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
gemini/gemini-live-2.5-flash-preview-native-audio-09-2025 uses mode='realtime'
but the schema in test_aaamodel_prices_and_context_window_json_is_valid did
not include 'realtime' as a valid enum value, causing a ValidationError.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(mcp): BYOK (Bring Your Own Key) for OpenAPI MCP servers with OAuth 2.1 flow
Adds per-user credential storage for BYOK MCP servers so external clients
can authenticate via standard OAuth 2.1 PKCE without needing a full identity
provider.
Backend:
- New DB table LiteLLM_MCPUserCredentials (user_id, server_id, credential_b64)
- is_byok, byok_description, byok_api_key_help_url fields on MCPServerTable
- OAuth 2.1 authorization server endpoints (/.well-known/oauth-authorization-server,
/.well-known/oauth-protected-resource, /v1/mcp/oauth/authorize, /v1/mcp/oauth/token)
- 401 challenge with WWW-Authenticate header when BYOK server has no credential
- CRUD endpoints: POST/DELETE /v1/mcp/server/{id}/user-credential
- has_user_credential annotated on GET /v1/mcp/server response
UI:
- ByokCredentialModal: 2-step Connect flow (access description + API key entry)
- BYOK toggle + description fields on admin MCP server create form
- Connect/Connected state in MCP server table
- BYOK Demo page (/tools/byok-demo) showing full OAuth 2.1 PKCE flow
* feat(mcp/byok): redesign OAuth authorize page to match 2-step Connect mockup
- Step 1: L→S logos, requested access checklist, How it works box, Continue button
- Step 2: API key input, Save toggle, Duration pills (1h/24h/7d/30d/until_revoked), security note
- Matches screenshots: white modal on dark bg, progress dots, dark CTA buttons
- Authorize handler now fetches byok_description and byok_api_key_help_url from server registry
- CLAUDE.md: replace SQL snippet with proper DB migration troubleshooting guidance
* fix: address greptile review feedback (greploop iteration 1)
- XSS: escape all user-supplied values in _build_authorize_html() with html.escape()
- Open redirect: validate redirect_uri scheme and URL-encode code/state in redirect
- N+1 query: batch BYOK credential lookup into single find_many() call
- Critical path DB: add 60s TTL in-memory cache to _check_byok_credential()
- Encrypt BYOK credentials at rest using encrypt_value_helper/decrypt_value_helper
* fix(byok): update OAuth popup with LiteLLM logo, MCP title suffix, remove emojis
* fix(byok-demo): fix token endpoint URL (/v1/mcp/oauth/token not /v1/mcp/token)
* feat(byok): inject stored BYOK credential as mcp_auth_header on tool execution
* feat(byok): use contextvars to inject per-user credential into OpenAPI tool closures; remove byok-demo from LiteLLM UI
OpenAPI tools have auth headers baked into their closures at registration time. BYOK servers have
no static auth token, so per-user credentials were never reaching the HTTP calls.
Fix: add _request_auth_header ContextVar in openapi_to_mcp_generator.py. create_tool_function now
reads this var at call time and overrides the Authorization header if set. execute_mcp_tool resolves
the MCP server and performs BYOK checks before the local-tool dispatch branch, then sets the
ContextVar around _handle_local_mcp_tool so the credential flows into the HTTP request.
Also remove the /tools/byok-demo page from the LiteLLM UI dashboard — the demo lives at
~/Downloads/litellm-byok-demo/index.html (served separately on port 8080).
* fix: address greptile review feedback (greploop iteration 2)
- Cache invalidation: add _invalidate_byok_cred_cache() and call it after
store_user_credential() in both token endpoint and management endpoint
- Unbounded cache: add _BYOK_CRED_CACHE_MAX_SIZE=4096 with clear-on-overflow
- Unbounded auth codes: add _AUTH_CODES_MAX_SIZE=1000 with 503 on overflow
- Double DB query: merge _check_byok_credential + _get_byok_credential into
single _get_byok_credential call; raise 401 inline if None returned
- Sidebar: remove byok-demo entry (page was deleted in prior commit)
- JWT comment: document why byok_session HS256 token can't be used as proxy auth
* fix: address greptile review feedback (greploop iteration 3)
- auth_type: pre-format Authorization header (Bearer/ApiKey/Basic) in server.py
before setting ContextVar so openapi_to_mcp_generator respects server auth_type
- cache invalidation on delete: call _invalidate_byok_cred_cache after
delete_user_credential so stale True entries don't persist for 60s
- ContextVar guard: only set _request_auth_header when mcp_auth_header is set,
avoiding unnecessary ContextVar overhead on non-BYOK tool calls
* fix: address greptile review feedback (greploop iteration 4)
- Unified credential cache: store actual credential value (Optional[str])
instead of just bool so _get_byok_credential also benefits from caching —
eliminates the DB hit on every BYOK tool call within the 60s TTL window
- Extracted _write_byok_cred_cache() helper for consistent cache writes
- Replaced has_user_credential with get_user_credential in _check_byok_credential
so one DB call satisfies both existence check and value retrieval
- Remove false 'encrypted at rest' claim from OAuth HTML and ByokCredentialModal
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_byok_oauth_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_byok_oauth_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
filter_helpers.ts was not populating the Organization ID filter dropdown
(always empty). TeamVirtualKeysTable was showing the team's org for all
keys instead of each key's own org.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The /key/list API returns `org_id` (the Pydantic field name), but the UI
was reading `organization_id`, causing the Organization field to always
show "Not Set" and the Organization ID filter to never match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a localStorage-based toggle to hide the bouncing 🌑 icon next to
the version tag in the navbar, following the same pattern used for
hiding prompts, usage indicator, new feature badges, and blog posts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove required validation for models field in create_key_button.tsx
- Update help text to clarify models are optional
- If no models selected, key will have access to all models
- This allows users to create keys for MCP-only access without selecting LLM models
- Fixes LIT-1791: Cannot create virtual key without LLM provider if user only has MCP access
Backend already supports empty models list (defaults to all models).
This change only updates the UI source to match backend behavior.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
- Added showSearch prop to enable search input in MCP server selector
- Added filterOption to search across server name, alias, server_id, and description
- Search is case-insensitive and filters in real-time
- Added test to verify search input appears when dropdown opens
- Updated tsconfig.json with Next.js auto-configuration (jsx: react-jsx)
Fixes issue where MCP server search was not working in the playground.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: complexity_router fails on list-format message content (OpenAI multi-part messages)
When a client sends messages with list-format content
(e.g. [{"type": "text", "text": "..."}] as used by the OpenAI JS SDK
and other clients), the complexity_router's async_pre_routing_hook
skipped those messages because it only handled str content. This caused
user_message to be None, the hook returned None, and the router fell
through to selecting the complexity_router deployment itself
(model="auto_router/complexity_router") which litellm cannot dispatch,
resulting in LiteLLMUnknownProvider.
Fixes:
- Extract text from list-format content parts (type=text) before
classifying
- Return default_model instead of None when no user message can be
extracted, preventing the crash fallthrough
- Loosen PreRoutingHookResponse.messages type from Dict[str, str] to
Dict[str, Any] to accommodate list-format content values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update messages type annotation in async_pre_routing_hook to Dict[str, Any]
Consistent with PreRoutingHookResponse.messages type change and the
list-format content support added in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: normalize None content to empty string in complexity_router message parsing
msg.get("content", "") returns None when the key exists with value None
(e.g. assistant messages with tool calls). Use `or ""` to normalize
None to an empty string explicitly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: strip whitespace from joined list content parts in complexity_router
Prevents leading/trailing spaces when some content parts have empty
text values (e.g. " ".join(["", "hello"]) → " hello").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The OpenAPI-to-MCP feature (PR #21575) added spec_path to the code
(_types.py, mcp_server_manager.py) but missed adding the column to
the Prisma schema files. This causes "Could not find field spec_path"
errors when creating OpenAPI-based MCP servers via the UI or API.
Adds `spec_path String?` to LiteLLM_MCPServerTable in all three
schema files (root, litellm/proxy, litellm-proxy-extras).
Made-with: Cursor
* (sap) ensure tool parameters have type='object' for SAP compatibility
Fix SAP GenAI Hub Orchestration Service rejecting tool calls with error:
"400 - LLM Module: tools.0.custom.input_schema.type: Input should be 'object'"
Root cause: When Claude Code uses tools (like web_search) with the SAP provider
through LiteLLM's Anthropic experimental pass-through adapter, Anthropic's
input_schema format doesn't always include the required type="object" field.
The adapter's translate_anthropic_tools_to_openai() function was directly
copying input_schema to OpenAI's parameters field without ensuring the
type="object" requirement that SAP's API strictly enforces.
Changes:
- Modified translate_anthropic_tools_to_openai() to check if input_schema
is missing the type field and add type="object" if absent
- Preserves existing type field if already present
- Added comprehensive test suite (6 tests) covering:
- Missing type field scenario (now adds type="object")
- Existing type preservation
- Empty input_schema handling
- Multiple tools transformation
- Additional schema properties preservation
- SAP-specific compatibility regression test
Testing:
- All new tests pass (6/6 in test_anthropic_tool_schema_fix.py)
- All existing Anthropic tool tests pass (57/57 tool-related tests)
- SAP tool parameter validation tests pass (9/9 in test_sap_tool_parameters.py)
* (sap) enable native response_format for anthropic models
* (sap) filter strict param from model_params for GPT models only
* (sap) revert Anthropic adapter type='object' fix
The SAP FunctionTool Pydantic validator in litellm/llms/sap/chat/models.py
already ensures type='object' is added to all tool parameters for SAP
API compatibility.
The Anthropic adapter change affected ALL consumers, not just SAP, which
was broader scope than intended for this PR.
- Revert input_schema modification in Anthropic adapter
- Remove Anthropic-specific test file (SAP tests still cover this case)
* (sap) gate markdown stripping to Anthropic models only
SAP GenAI Hub with Anthropic models sometimes returns JSON wrapped in
markdown code blocks. GPT/Gemini/Mistral models don't exhibit this
behavior, so stripping is now gated to avoid accidentally modifying
valid responses that may contain markdown in JSON string values.
* fix(proxy): add guardrails list routes for internal users
* fix(ui): add guardrails fetch with v1/v2 fallback in networking
* fix(ui): allow internal users/team admins to select guardrails in create key modal
* fix(ui): show guardrails selector for internal users in key edit view
* fix(ui): pass canEditGuardrails flag to key info view
* test(ui): add tests for role-based guardrails access in key info view
* test(ui): update key edit view test for guardrails
