Commit Graph

7177 Commits

Author SHA1 Message Date
Julio Quinteros Pro
e8301829cd Fix flaky MCP streaming test by properly mocking inner aresponses call
The test_streaming_mcp_events_validation test was flaky because:
1. It didn't mock the nested aresponses() call inside the iterator's
   _create_initial_response_iterator(), causing real API calls that fail
   without credentials
2. The iterator silently swallowed exceptions and set phase="finished",
   discarding pre-generated MCP discovery events
3. The _execute_tool_calls mock had wrong signature (missing tool_server_map)

Production fix: MCPEnhancedStreamingIterator no longer sets phase="finished"
on LLM call failure — it falls through to emit MCP discovery events first.

Test fix: Added mock for litellm.responses.main.aresponses returning a fake
async streaming iterator, fixed mock signatures, removed try/except that
masked failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 11:09:24 -03:00
Sameer Kankute
db0d5588fa
Merge pull request #22740 from BerriAI/litellm_fix_file_wild_card
Add support for wildcards models for files api
2026-03-04 18:30:00 +05:30
Sameer Kankute
ece7fdb213
Merge pull request #22744 from BerriAI/litellm_mcp_streaming_fix
Add mcp streaming events Fix and consistent response ID
2026-03-04 18:29:48 +05:30
Sameer Kankute
b5183e9f3b
Merge pull request #22752 from BerriAI/litellm_search_api_add
[Feat] Add Google Search API Integration
2026-03-04 18:29:10 +05:30
Sameer Kankute
213bf11ede
Merge pull request #22763 from BerriAI/litellm_test_e2e_batches_test
feat(tests): add proxy e2e azure batches test
2026-03-04 18:28:52 +05:30
Sameer Kankute
8764e5da8c
Merge pull request #22559 from BerriAI/litellm_responses_websocket
[Feat] Add support for Responses Websocket
2026-03-04 17:57:34 +05:30
Sameer Kankute
4e229b7b21
Revert "fix(anthropic): remove hardcoded reasoning summary in adapter" 2026-03-04 17:52:49 +05:30
Sameer Kankute
7d790b39be
Merge pull request #22765 from BerriAI/main
merge main for 030326
2026-03-04 17:40:42 +05:30
Sameer Kankute
812e455cbc feat(tests): add proxy e2e azure batches test cases
- test_proxy_e2e_azure_batches: e2e managed batch test with delete retry for batch_processed
- test_fixtures_smoke: smoke test for fixtures
- validate_e2e_setup: setup validation script

Made-with: Cursor
2026-03-04 17:15:30 +05:30
Sameer Kankute
52693a65ed feat(tests): add proxy e2e azure batches test infrastructure
- conftest: mock server + proxy server fixtures, log capture, health check fix
- base_integration_test: fix key_alias format (replace @ and . for API validation)
- test_managed_files_base: S3 callback wait with early exit, delete retry logic

Made-with: Cursor
2026-03-04 17:15:24 +05:30
Sameer Kankute
e3a93b9597 feat(tests): add proxy e2e azure batches test fixtures
- Mock Azure Batch server (FastAPI) with batch, file, chat, embeddings, S3 callback routes
- Config for mock models and batch polling
- Fix mock_responses: use 'model' instead of undefined 'request_model'

Made-with: Cursor
2026-03-04 17:15:15 +05:30
Harshit Jain
41b149ee93
Merge pull request #22678 from Harshit28j/litellm_custom_auth_opt_in
fix(proxy): make common_checks opt-in for custom auth
2026-03-04 14:53:44 +05:30
Sameer Kankute
43db8f1f1e Add doc and tests for google search api 2026-03-04 13:55:43 +05:30
Sameer Kankute
0275e23601 Add routing for google search 2026-03-04 13:54:43 +05:30
Sameer Kankute
f878923d26 Add test for correct streaming chunks and responses id consistency 2026-03-04 12:19:54 +05:30
Sameer Kankute
dc2d465c7e Add support for wildcards models for files api 2026-03-04 11:48:42 +05:30
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693)
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
2026-03-03 20:27:05 -08:00
Ishaan Jaff
7befe3c78f
feat(proxy): add key_alias, key_hash, requested_model DD APM span tags (#22710)
* feat(proxy): add key_alias, key_hash, requested_model tags to DD APM spans

* refactor(proxy): consolidate DD APM tag helpers into DDSpanTagger class

* refactor(proxy): move DDSpanTagger to its own file litellm/proxy/dd_span_tagger.py
2026-03-03 20:22:59 -08:00
Ishaan Jaff
1f412bc6d8
[Feat] Add Tool Policies for AI Gateway (#22732)
* fix: fix ui render

* fix: fix minor bugs

* refactor: use prisma functions instead of raw sql (safer)

* fix(add-new-tiles-to-tool-policies): allow developer to see what's available

* feat: ensure tool allowlist runs correctly for tool names + mcp's

* refactor: more ui improvements

* feat: working key tool blocking

* feat(tools): show tool logs

* refactor: backend code improvements

* refactor: improve log viewer for tools

* fix: address PR review feedback for tool access control

- Add missing blocked_tools column to root schema.prisma (schema drift)
- Invalidate ToolPolicyRegistry after policy mutations so changes take effect immediately
- Remove dead code: unused get_effective_policies, get_tool_policies_cached, and helpers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: race condition in permission resolution and remove duplicate allowlist check

- Use atomic update_many with object_permission_id=None to prevent concurrent
  requests from creating orphaned permission rows and losing tool blocks
- Remove duplicate allowed_tools enforcement from guardrail (already enforced
  in auth layer via check_tools_allowlist)
- Move inline uuid import to module level

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update to account for  userAgent

* UI - Add ToolDetails

* input/output policy

* LiteLLM_PolicyAttachmentTable

* LiteLLM_PolicyAttachmentTable

* fix: add _enqueue_tool_registry_upsert

* fix: tool mgmt endpoints

* tool mgmt endpoints

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: sync root schema.prisma and fix test_tool_registry_writer for input/output policy

- Migrate root schema.prisma LiteLLM_ToolTable from call_policy to
  input_policy/output_policy, add missing user_agent and last_used_at columns
  (now consistent with litellm/proxy/schema.prisma and litellm-proxy-extras)
- Fix SpendLogToolIndex comment across all three schema files
- Fix all call_policy references in test_tool_registry_writer.py:
  swapped update_tool_policy arguments, wrong get_tools_by_names return type
  assertions, _mock_tool_row setting call_policy instead of input_policy

Addresses Greptile review feedback on PR #22732.

Made-with: Cursor

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 20:22:20 -08:00
Sameer Kankute
120201049e
Merge pull request #22666 from Point72/ephrimstanley/batch-fixes-mar3
Managed batches - Address PR bot comments from #22464
2026-03-04 09:06:41 +05:30
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling
fix: add missing supports_function_calling for deepinfra models
2026-03-04 08:51:21 +05:30
Krish Dholakia
90eb6729d5
Agent Tracing - support context_id based trace id propogation + nested llm calls (#22626)
* style(ui/): distinguish agent calls from llm calls on ui

* feat: initial grouping working

* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls

* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call

allows stable trace id usage

* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict

_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: propagate trace/session id from headers in MCP server calls

Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:19:12 -08:00
Cesar Garcia
4ab79451d9
Merge pull request #22552 from shanemort1982/fix/register-model-custom-pricing-fields
fix: pass all custom pricing fields to register_model in completion() and embedding()
2026-03-03 22:40:45 -03:00
yuneng-jiang
ab101671c7
Merge pull request #22725 from BerriAI/litellm_team_admin_reset_spend
[Fix] Team Admin Reset Key Spend
2026-03-03 16:50:46 -08:00
yuneng-jiang
0a1b2635d7 fix: allow team admins to access /key/{key}/reset_spend route
The route-level auth check was blocking internal_user role (team admins)
from reaching /key/{key}/reset_spend because KEY_RESET_SPEND was missing
from key_management_routes. Added it so team admins pass the route check
and the endpoint's existing _check_proxy_or_team_admin_for_key enforces
actual authorization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:34 -08:00
milan-berri
98b9bc8b72
fix: resolve base_model in /cost/estimate for Azure custom deployments (#22724)
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.

Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
2026-03-03 15:43:02 -08:00
Julio Quinteros Pro
d9a349f7d2
Merge pull request #22720 from BerriAI/fix/proxy-utils-mock-request-state
fix: use real State in mock_request for proxy_utils tests
2026-03-03 20:10:53 -03:00
Julio Quinteros Pro
740cdc5c20 fix: use real State object in mock_request to fix _safe_get_request_headers
The _safe_get_request_headers caching (commit e7175a52) uses
request.state._cached_headers. With Mock(spec=Request), getattr on
state returns a Mock (truthy), causing RedactedDict to receive a Mock
instead of a dict. Using a real starlette State object fixes this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:05:20 -03:00
Julio Quinteros Pro
2415035fcd
Merge pull request #22719 from BerriAI/fix/vertex-response-format-test
fix: account for vertex_ai schema diff in response_format test
2026-03-03 19:59:24 -03:00
Julio Quinteros Pro
5dfa96f62f
Merge pull request #22718 from BerriAI/fix/jwt-test-mock-robustness
fix: use direct AsyncMock in JWT OIDC discovery tests
2026-03-03 19:59:15 -03:00
Julio Quinteros Pro
a07d041881 fix: apply same AsyncMock pattern to remaining OIDC discovery test
Address Greptile review: test_resolve_jwks_url_resolves_oidc_discovery_document
also used the inconsistent patch.object pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:56:51 -03:00
Julio Quinteros Pro
9b92ea16ab fix: update response_format test for vertex_ai's intentional schema diff
Vertex AI / Gemini uses Pydantic's model_json_schema() which omits
additionalProperties: False (Gemini rejects it). The test expected
the same schema for all providers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:55:18 -03:00
Julio Quinteros Pro
eb658693a3 fix: use direct AsyncMock assignment instead of patch.object in JWT tests
The patch.object with new_callable=AsyncMock can behave inconsistently
across Python versions, causing mock_response.status_code to return a
MagicMock instead of the assigned value. Direct assignment is simpler
and more reliable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:51:09 -03:00
Julio Quinteros Pro
e6a1194c28
Merge pull request #22717 from BerriAI/fix/types-utils-video-tokens-test
fix: add video_tokens to completion_tokens_details test
2026-03-03 19:48:44 -03:00
Julio Quinteros Pro
2943df96ba fix: add video_tokens to expected completion_tokens_details in test
The CompletionTokensDetailsWrapper type now includes video_tokens field,
but this test's expected dict was not updated to include it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:46:20 -03:00
Julio Quinteros Pro
fc9d06ceca
Merge pull request #22716 from BerriAI/fix/vertex-function-response-tests
fix: update vertex AI tests for function_response role=user
2026-03-03 19:44:54 -03:00
Julio Quinteros Pro
5a0aba9fb7 fix: update vertex AI tests to expect role=user on function_response messages
The Gemini API requires role="user" on function_response content blocks
(added in commit 273cf12afa), but these tests were never updated to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:40:28 -03:00
Julio Quinteros Pro
2c5c38333d
Merge pull request #22701 from BerriAI/fix/streaming-and-azure-gpt5-test-failures
Fix Anthropic streaming sync and Azure GPT-5.1 logprobs tests
2026-03-03 19:36:44 -03:00
ryan-crabbe
8aa7d53864
Merge pull request #22705 from BerriAI/litellm_feat_batch_expiry_setting_teams
Litellm feat batch expiry setting teams
2026-03-03 14:36:12 -08:00
Julio Quinteros Pro
6b4bc99202 Fix Anthropic streaming sync __next__ and Azure GPT-5.1 logprobs
Two independent fixes for pre-existing test failures on main:

1. Anthropic streaming: The sync __next__ method used a simple
   holding_chunk pattern that lost chunks when multiple events needed
   to be returned. Refactored to use the same chunk_queue approach as
   the async __anext__ method. Also fixed tests that used ModelResponse
   (which defaults finish_reason to 'stop') instead of ModelResponseStream.

2. Azure GPT-5.1 logprobs: The base OpenAI class includes logprobs for
   gpt-5.1+ models, but Azure hasn't verified support for gpt-5.1.
   Added explicit removal of logprobs/top_logprobs for gpt-5.1 (non-5.2)
   models in the Azure config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:16:11 -03:00
Harshit Jain
b62a4be6b6
Merge pull request #22676 from BerriAI/litellm_feat_guardrail-list
feat: guardrail-mode-default-list
2026-03-04 02:36:57 +05:30
Harshit28j
d661419109 fix: support list of modes in Mode.default for tag-based guardrails 2026-03-04 01:50:28 +05:30
Ryan Crabbe
08613b24cb feat(proxy): add team-level file expiry enforcement 2026-03-03 11:03:14 -08:00
Ryan Crabbe
3d15bcdb11 feat(proxy): add team-level batch output expiry enforcement 2026-03-03 10:58:31 -08:00
yuneng-jiang
4c1b15d685
Merge pull request #22687 from BerriAI/litellm_fix_spend_log_cleanup
[Fix] Spend Log Cleanup: lock tracking, integer retention, skip log level
2026-03-03 10:48:20 -08:00
Ryan Crabbe
43cec8c980 feat(batches): support output_expires_after passthrough 2026-03-03 10:46:45 -08:00
Cesar Garcia
8a1e915bb6
Merge pull request #22403 from shivaaang/fix/openrouter-image-edit
feat(openrouter): add image edit support for OpenRouter models
2026-03-03 15:31:04 -03:00
Cesar Garcia
d384f7c320
Merge pull request #21233 from Chesars/feat/per-request-json-schema-validation
feat: support per-request enable_json_schema_validation for thread safety
2026-03-03 15:29:54 -03:00
Cesar Garcia
a8b5a876bf
Merge pull request #21491 from Chesars/fix/20998-remove-hardcoded-reasoning-summary
fix(anthropic): remove hardcoded reasoning summary in adapter
2026-03-03 15:29:11 -03:00
Chesars
833c1bc45d merge: resolve conflict with staging, remove hardcoded summary from reasoning test 2026-03-03 15:28:45 -03:00