Commit Graph

34306 Commits

Author SHA1 Message Date
Sameer Kankute
ece7fdb213
Merge pull request #22744 from BerriAI/litellm_mcp_streaming_fix
Add mcp streaming events Fix and consistent response ID
2026-03-04 18:29:48 +05:30
Sameer Kankute
b5183e9f3b
Merge pull request #22752 from BerriAI/litellm_search_api_add
[Feat] Add Google Search API Integration
2026-03-04 18:29:10 +05:30
Sameer Kankute
213bf11ede
Merge pull request #22763 from BerriAI/litellm_test_e2e_batches_test
feat(tests): add proxy e2e azure batches test
2026-03-04 18:28:52 +05:30
Sameer Kankute
8764e5da8c
Merge pull request #22559 from BerriAI/litellm_responses_websocket
[Feat] Add support for Responses Websocket
2026-03-04 17:57:34 +05:30
github-actions[bot]
09ee2d54b8
chore: regenerate poetry.lock to match pyproject.toml (#22769)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-04 12:24:44 +00:00
Sameer Kankute
a6b5197ddc
Merge pull request #22689 from BerriAI/litellm_oss_staging_03_03_2026
Litellm oss staging 03 03 2026
2026-03-04 17:53:57 +05:30
Sameer Kankute
95c6a7e183
Merge pull request #22768 from BerriAI/revert-21491-fix/20998-remove-hardcoded-reasoning-summary
Revert "fix(anthropic): remove hardcoded reasoning summary in adapter"
2026-03-04 17:53:02 +05:30
Sameer Kankute
4e229b7b21
Revert "fix(anthropic): remove hardcoded reasoning summary in adapter" 2026-03-04 17:52:49 +05:30
Sameer Kankute
7d790b39be
Merge pull request #22765 from BerriAI/main
merge main for 030326
2026-03-04 17:40:42 +05:30
Sameer Kankute
d14420b8f2 Add tenacity in dependencies 2026-03-04 17:28:22 +05:30
Sameer Kankute
ef7aa6c8ad Add this test in cicd 2026-03-04 17:24:48 +05:30
Sameer Kankute
7b6a972fed Add this test in cicd 2026-03-04 17:21:00 +05:30
Sameer Kankute
49738bb3e3 ci: add proxy e2e azure batches workflow
- Run test_e2e_managed_batch with -vv -s for terminal output on failure
- PostgreSQL, Poetry, Prisma setup
- Upload logs as artifact on failure

Made-with: Cursor
2026-03-04 17:15:33 +05:30
Sameer Kankute
812e455cbc feat(tests): add proxy e2e azure batches test cases
- test_proxy_e2e_azure_batches: e2e managed batch test with delete retry for batch_processed
- test_fixtures_smoke: smoke test for fixtures
- validate_e2e_setup: setup validation script

Made-with: Cursor
2026-03-04 17:15:30 +05:30
Sameer Kankute
52693a65ed feat(tests): add proxy e2e azure batches test infrastructure
- conftest: mock server + proxy server fixtures, log capture, health check fix
- base_integration_test: fix key_alias format (replace @ and . for API validation)
- test_managed_files_base: S3 callback wait with early exit, delete retry logic

Made-with: Cursor
2026-03-04 17:15:24 +05:30
Sameer Kankute
e3a93b9597 feat(tests): add proxy e2e azure batches test fixtures
- Mock Azure Batch server (FastAPI) with batch, file, chat, embeddings, S3 callback routes
- Config for mock models and batch polling
- Fix mock_responses: use 'model' instead of undefined 'request_model'

Made-with: Cursor
2026-03-04 17:15:15 +05:30
Harshit Jain
41b149ee93
Merge pull request #22678 from Harshit28j/litellm_custom_auth_opt_in
fix(proxy): make common_checks opt-in for custom auth
2026-03-04 14:53:44 +05:30
Harshit28j
a6b57f9301 doc: fix typo 2026-03-04 14:52:27 +05:30
Sameer Kankute
43db8f1f1e Add doc and tests for google search api 2026-03-04 13:55:43 +05:30
Sameer Kankute
0275e23601 Add routing for google search 2026-03-04 13:54:43 +05:30
Sameer Kankute
65eba6b70e Add search api config 2026-03-04 13:54:25 +05:30
yuneng-jiang
d0e30842e5
Merge pull request #21958 from atapia27/feat/virtual-keys-team-table
virtual-keys-team-table
2026-03-03 23:01:34 -08:00
Sameer Kankute
9907c635ef Fix: Removed the process-level _encrypted_response_id_cache from __init__ 2026-03-04 12:30:49 +05:30
yuneng-jiang
e331d86f26
Merge pull request #22316 from atapia27/reusable-credential-change
reusable-credentials
2026-03-03 22:59:54 -08:00
Sameer Kankute
f878923d26 Add test for correct streaming chunks and responses id consistency 2026-03-04 12:19:54 +05:30
Sameer Kankute
e0880c3dee Fix order of streaming for mcp responses 2026-03-04 12:11:16 +05:30
yuneng-jiang
2f192e8d97
Merge pull request #22739 from BerriAI/litellm_ui_build_fix_march3
[Fix] UI - Tool Policies: Add missing ToolRow properties
2026-03-03 22:19:17 -08:00
yuneng-jiang
baca1937aa fixing build 2026-03-03 22:15:56 -08:00
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693)
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
2026-03-03 20:27:05 -08:00
Ishaan Jaff
7befe3c78f
feat(proxy): add key_alias, key_hash, requested_model DD APM span tags (#22710)
* feat(proxy): add key_alias, key_hash, requested_model tags to DD APM spans

* refactor(proxy): consolidate DD APM tag helpers into DDSpanTagger class

* refactor(proxy): move DDSpanTagger to its own file litellm/proxy/dd_span_tagger.py
2026-03-03 20:22:59 -08:00
Ishaan Jaff
1f412bc6d8
[Feat] Add Tool Policies for AI Gateway (#22732)
* fix: fix ui render

* fix: fix minor bugs

* refactor: use prisma functions instead of raw sql (safer)

* fix(add-new-tiles-to-tool-policies): allow developer to see what's available

* feat: ensure tool allowlist runs correctly for tool names + mcp's

* refactor: more ui improvements

* feat: working key tool blocking

* feat(tools): show tool logs

* refactor: backend code improvements

* refactor: improve log viewer for tools

* fix: address PR review feedback for tool access control

- Add missing blocked_tools column to root schema.prisma (schema drift)
- Invalidate ToolPolicyRegistry after policy mutations so changes take effect immediately
- Remove dead code: unused get_effective_policies, get_tool_policies_cached, and helpers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: race condition in permission resolution and remove duplicate allowlist check

- Use atomic update_many with object_permission_id=None to prevent concurrent
  requests from creating orphaned permission rows and losing tool blocks
- Remove duplicate allowed_tools enforcement from guardrail (already enforced
  in auth layer via check_tools_allowlist)
- Move inline uuid import to module level

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update to account for  userAgent

* UI - Add ToolDetails

* input/output policy

* LiteLLM_PolicyAttachmentTable

* LiteLLM_PolicyAttachmentTable

* fix: add _enqueue_tool_registry_upsert

* fix: tool mgmt endpoints

* tool mgmt endpoints

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update tests/test_litellm/proxy/db/test_tool_registry_writer.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: sync root schema.prisma and fix test_tool_registry_writer for input/output policy

- Migrate root schema.prisma LiteLLM_ToolTable from call_policy to
  input_policy/output_policy, add missing user_agent and last_used_at columns
  (now consistent with litellm/proxy/schema.prisma and litellm-proxy-extras)
- Fix SpendLogToolIndex comment across all three schema files
- Fix all call_policy references in test_tool_registry_writer.py:
  swapped update_tool_policy arguments, wrong get_tools_by_names return type
  assertions, _mock_tool_row setting call_policy instead of input_policy

Addresses Greptile review feedback on PR #22732.

Made-with: Cursor

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 20:22:20 -08:00
Ishaan Jaff
8baa3ae8cb
[Feat] UI - Add Open in New Tab on leftnav Bar (#22731)
* Add minimal dev_config.yaml for proxy development

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): wrap left nav items in <a> tags for open-in-new-tab support

Nav items are now rendered as <a> elements with proper href attributes,
enabling right-click → 'Open in new tab', Ctrl/Cmd+click, and
middle-click to open any sidebar page in a new browser tab.

Normal clicks continue to use SPA navigation (no full page reload).

Applied to both leftnav.tsx (query-param routing) and Sidebar2.tsx
(Next.js file-based routing).

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-03 19:56:55 -08:00
Sameer Kankute
120201049e
Merge pull request #22666 from Point72/ephrimstanley/batch-fixes-mar3
Managed batches - Address PR bot comments from #22464
2026-03-04 09:06:41 +05:30
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling
fix: add missing supports_function_calling for deepinfra models
2026-03-04 08:51:21 +05:30
Sameer Kankute
8541272629
Merge pull request #22685 from Varad2001/litellm_support_Qwen3.5-397B-A17B
feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model
2026-03-04 08:31:49 +05:30
Krish Dholakia
90eb6729d5
Agent Tracing - support context_id based trace id propogation + nested llm calls (#22626)
* style(ui/): distinguish agent calls from llm calls on ui

* feat: initial grouping working

* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls

* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call

allows stable trace id usage

* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict

_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: propagate trace/session id from headers in MCP server calls

Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:19:12 -08:00
Cesar Garcia
4ab79451d9
Merge pull request #22552 from shanemort1982/fix/register-model-custom-pricing-fields
fix: pass all custom pricing fields to register_model in completion() and embedding()
2026-03-03 22:40:45 -03:00
yuneng-jiang
ba7a6d9bfd
Merge pull request #22476 from BerriAI/litellm_audit_pagination_fix
[Fix] UI - Audit Logs: Server-side pagination, filtering, and drawer view
2026-03-03 16:52:33 -08:00
yuneng-jiang
ab101671c7
Merge pull request #22725 from BerriAI/litellm_team_admin_reset_spend
[Fix] Team Admin Reset Key Spend
2026-03-03 16:50:46 -08:00
Alejandro Tapia
5f38c3bfec
Update ui/litellm-dashboard/src/components/team/TeamVirtualKeysTable.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 16:30:10 -08:00
Alejandro Tapia
18a2213cdb Fix: Promise.all loses all results on single page failure 2026-03-03 16:21:02 -08:00
Alejandro Tapia
34acbb39da
Update ui/litellm-dashboard/src/components/team/TeamVirtualKeysTable.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 16:19:32 -08:00
Alejandro Tapia
47ff0d1d86 Fixed: 50 sequential API calls on mount is excessive 2026-03-03 16:03:25 -08:00
Alejandro Tapia
383f79f36f
Update ui/litellm-dashboard/src/components/key_team_helpers/filter_helpers.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 16:00:57 -08:00
yuneng-jiang
5fa0c6e994 fix(ui): Copyable JSON blocks in audit drawer; custom table spinner
- Replace plain JsonBlock with CopyableJsonBlock: header row with label
  and copy icon (CheckOutlined on success), pre block below — matches
  the spend logs RequestResponsePanel pattern
- Key-table rows use the same card chrome for visual consistency
- Remove separate "Changes" section label (now redundant with block headers)
- Table loading spinner replaced with custom Spin + LoadingOutlined

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:00:46 -08:00
Alejandro Tapia
51f18b05a9
Update ui/litellm-dashboard/src/components/team/TeamVirtualKeysTable.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 16:00:16 -08:00
Alejandro Tapia
474eea5a1b
Merge branch 'main' into feat/virtual-keys-team-table 2026-03-03 15:56:19 -08:00
yuneng-jiang
326ff422f1 fix(ui): Audit logs table and drawer polish
Table:
- Use DefaultProxyAdminTag for changed_by column
- Remove tooltips on Object ID and API Key columns
- Rename API Key column to API Key (Hash)
- Move pagination controls to upper-right of filter bar; add icon-only refresh button

Drawer:
- Object ID is now copyable
- API Key (Hash) is copyable and no longer truncated
- Changed By uses DefaultProxyAdminTag
- Expand JSON view boxes from max-h-72 to max-h-96
- Remove unnecessary vertical scrollbar (drop overflow-auto h-full from body div;
  use flex column layout so header and content flow naturally)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:46 -08:00
yuneng-jiang
0a1b2635d7 fix: allow team admins to access /key/{key}/reset_spend route
The route-level auth check was blocking internal_user role (team admins)
from reaching /key/{key}/reset_spend because KEY_RESET_SPEND was missing
from key_management_routes. Added it so team admins pass the route check
and the endpoint's existing _check_proxy_or_team_admin_for_key enforces
actual authorization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:34 -08:00
milan-berri
98b9bc8b72
fix: resolve base_model in /cost/estimate for Azure custom deployments (#22724)
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.

Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
2026-03-03 15:43:02 -08:00