Commit Graph

26963 Commits

Author SHA1 Message Date
RoniGurvichCycode
eeda5ab84e
requirements update (#16054)
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-10-31 19:18:03 -07:00
Ishaan Jaffer
6be9b97d15 test_stream_token_counting_anthropic_with_include_usage 2025-10-31 19:15:07 -07:00
dependabot[bot]
1bc88b5fc9
build(deps): bump starlette from 0.47.2 to 0.49.1 (#16027)
Bumps [starlette](https://github.com/Kludex/starlette) from 0.47.2 to 0.49.1.
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.47.2...0.49.1)

---
updated-dependencies:
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-31 19:13:11 -07:00
Ishaan Jaffer
6e46824939 test_streaming_response 2025-10-31 19:10:38 -07:00
Ishaan Jaffer
515ae9680e test_anthropic_tool_use 2025-10-31 19:08:32 -07:00
Ishaan Jaffer
f173cb0f54 test_anthropic_text_editor 2025-10-31 19:07:46 -07:00
Ishaan Jaffer
928fddb3be test_anthropic_structured_output 2025-10-31 19:06:45 -07:00
Ishaan Jaffer
7f79abb552 test_aastreaming_tool_calls_valid_json_str 2025-10-31 19:05:31 -07:00
Ishaan Jaffer
2cd57540a4 fix test fixes 2025-10-31 18:31:00 -07:00
Ishaan Jaffer
e6c9df230f fix linting 2025-10-31 18:26:22 -07:00
Ishaan Jaffer
22eb2f8033 Revert "Python entry-point for CustomLLM subclasses (#15881)"
This reverts commit 559ae96e38.
2025-10-31 18:24:39 -07:00
Ishaan Jaffer
94c2c28f3d claude-sonnet-4-5-20250929 fix 2025-10-31 18:20:52 -07:00
Ishaan Jaffer
bd52cb2983 bump proxy extras 2025-10-31 18:19:28 -07:00
Ishaan Jaffer
98af05fb43 add LiteLLM_SSOConfig 2025-10-31 18:15:37 -07:00
Ishaan Jaffer
159db27d5c fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
Ishaan Jaffer
d90f1696c3 UI new build 2025-10-31 18:11:41 -07:00
Ishaan Jaffer
16a1b57dd5 TestTavilySearch 2025-10-31 18:06:01 -07:00
Ishaan Jaffer
51349cdc99 fix _strip_base64_from_messages 2025-10-31 17:59:13 -07:00
Ishaan Jaffer
2fe2704a21 fixes code QA check 2025-10-31 17:55:49 -07:00
Ishaan Jaffer
73f745ee09 fix claude-sonnet-4-5-20250929 2025-10-31 17:48:49 -07:00
Ishaan Jaffer
09e18ea0ac fix get_guardrail_info 2025-10-31 17:45:38 -07:00
Ishaan Jaff
c59a0c9681
[Feat] UI - Allow setting cache settings on UI (#16143)
* add LiteLLM_CacheConfig

* add CacheSettingsField

* add UI cache saver

* feat add cache_settings_router

* fix schema

* fix ssl_check_hostname

* refactor into utils

* add groups for field names

* add test_connection in base cache

* add test_connection inredis and redis cluster

* feat _decrypt_db_variables

* add cache settings endpoints

* test_test_cache_connection_calls_cache_test_connection_with_params

* fix: add switch_on_llm_response_caching

* feat use CacheSettingsManager

* feat use CacheSettingsManager

* TestCacheSettingsManager

* fix update_config

* Cache Field test
2025-10-31 17:43:59 -07:00
yuneng-jiang
725671395a
[Feature] UI - Config Guardrails should not be editable and guardrail info fix (#16142)
* UI Config Guardrails should not be editable and guardrails info definition location fix

* Remove unused import

* Added literals for guardrail definition location
2025-10-31 16:29:24 -07:00
yuneng-jiang
cf0d84c789
[Feature] UI - Add Tags to update model (#16140)
* Add Tags to update model

* Test Key Page Tag Selector
2025-10-31 16:28:46 -07:00
yuneng-jiang
dd053e1d12
[Feature] UI - Add tags in model creation (#16138)
* Add tags in model creation

* Removing unused import in test
2025-10-31 16:26:30 -07:00
Ishaan Jaff
b6312e68f0
[Feat] UI + Management - Allow setting all routing strategies, tag filtering on UI (#16139)
* add ROUTER_SETTINGS_FIELDS

* feat: add router_settings_router on proxy

* add RouterSettingsField

* update RouterSettingsResponse

* add ROUTER_SETTINGS_FIELDS

* add getRouterSettingsCall

* add UI friendly name

* fix router settings

* fix get descriptions

* fix toggles /refator

* fix refactor

* fix refactor

* fix ROUTING_STRATEGY_DESCRIPTIONS

* fix fields
2025-10-31 11:29:37 -07:00
Raghav Jhavar
1f10e8e49c
make debug log debug (#16137) 2025-10-31 11:17:24 -07:00
1stprinciple
2824388c30
Don't add "accounts/fireworks/models" prefix for Fireworks Provider (#15938)
* init

* if not model.startswith("accounts/") and "#" not in model:
2025-10-30 20:36:42 -07:00
Sameer Kankute
689b210c18
Add Xai websearch cost (#16001)
* Add xai websearch cost

* Add test for websearch cost in xai

* remove not required changes
2025-10-30 20:35:34 -07:00
Boxuan Li
297c2a0bf4
prompt_caching.md: Fix wrong prompt_tokens definition (#16044) 2025-10-30 20:13:13 -07:00
Nicholas Couture
f747a4a38a
fix: Handle multiple rate limit types per descriptor and prevent IndexError (#16039)
* improve descriptor_key handling for multiple and missing rate limit descriptors in parallel request limiter v3

* Add tests for parallel request limiter v3 in proxy hooks
2025-10-30 20:12:54 -07:00
Sameer Kankute
c1369a07ba
Add Add per model group header forwarding for Bedrock Invoke API (#16042) 2025-10-30 20:10:17 -07:00
tanvithakur
bb79a75a89
Changes to fix frequency_penalty and presence_penalty issue for gemini-2.5-pro model (#16041)
* Update vertex_and_google_ai_studio_gemini.py

* Fix logic

* Fix mypy error

* Fix lgic

---------

Co-authored-by: tanvi thakur <tanvi_thakur@intuit.com>
2025-10-30 20:02:58 -07:00
John Lathouwers
a6f740f28b
Add OCI Signer Authentication. Closes #16048, Closes #15654 (#16064)
* Add OCI Signer Authentication. Closes #16048,  Closes #15654

* Fix linting error

* Remove Recommended, Catch None, Trim Whitespace

* Make method clear re Proxy vs SDK
2025-10-30 19:59:01 -07:00
pale-aura
0d84c11bbe
Fix vercel_ai_gateway entry for glm-4.6 (#16084)
* Fix vercel_ai_gateway entry for glm-4.6

* Fix: add unit test for vercel_ai_gateway/zai/glm-4.6

* Fix:

Add fix for unit test on vercel_ai_gateway/zai/glm-4.6

* Remove print statements in test_vercel_ai_gateway_glm46_cost_math
2025-10-30 19:40:35 -07:00
Geoffray Viossat
b29ba9e3be
feat: handle Deepgram detected language when available (#16093) 2025-10-30 19:19:34 -07:00
Daniel Dimenshtein
cde9ba5f93
Update model_prices_and_context_window.json, qwen-vl models (#16106)
Add supports vision, to qwen vl models
2025-10-30 19:18:35 -07:00
Ishaan Jaffer
4d88f21393 docs fix 2025-10-30 18:25:51 -07:00
Seongbin Lim
18e7eaaddb
Add gemini/gemini-embedding-001 pricing entry for Google GenAI API (#16078)
* Add gemini/gemini-embedding-001 pricing entry for Google GenAI API

Add pricing and configuration entry for "gemini/gemini-embedding-001"
model when called via Google GenAI API (google-genai provider). This
mirrors the existing Vertex AI entry ("gemini-embedding-001") but uses
the 'gemini' provider instead.

Co-Authored-By: Claude <noreply@anthropic.com>

* Update tpm and rpm for gemini/gemini-embedding-001

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-30 17:39:19 -07:00
Ishaan Jaff
0428229032
[Docs] readme fixes add supported providers (#16109)
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config

* add ENDPOINT_COLUMNS

* add provider endpoints

* docs fix
2025-10-30 17:23:20 -07:00
Krrish Dholakia
30f343c3cb docs(deploy.md): document new terraform module 2025-10-30 17:11:55 -07:00
Ishaan Jaff
fd073ee682
Litellm docs readme fixes (#16107)
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config
2025-10-30 17:05:32 -07:00
Ishaan Jaffer
40c663439a fix readme.md 2025-10-30 15:54:50 -07:00
yuneng-jiang
720ba865fb
[Infra] Litellm Backend SSO Changes (#16029)
* SSO Backend changes

* Encrypt and Decrypt, load into os env

* Linting and addressing comments
2025-10-30 14:32:08 -07:00
Sameer Kankute
eed3ad0bdb
Fix: Moderations endpoint now respects api_base configuration parameter (#16087)
* Update moderation to use api base

* Update moderation to use api base

* Fix mypy error
2025-10-30 11:01:26 -07:00
yuneng-jiang
b9850accaa
Circle CI e2e test fix (#16100) 2025-10-30 09:54:46 -07:00
Thomas Mildner
6fc33add9a
Opik user auth key metadata Documentation (#16004)
* fix(opik): enhance requester metadata retrieval from API key auth

* docs(opik): add automatic metadata association from API keys in documentation
2025-10-29 19:13:19 -07:00
Sameer Kankute
192935120d
Add OpenAI-compatible annotations support for Cohere v2 citations 2025-10-29 19:12:17 -07:00
Talal
5e10ea4136
Improve(mcp): respect X-Forwarded- headers in OAuth endpoints (#16036)
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints

When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.

Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix formatting

* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction

Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.

Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com

Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints

Fixes issue where protected resource URL mismatch occurred:
  Error: Protected resource http://proxy.abc.com:8888/github/mcp
  does not match expected https://proxy.abc.com/github/mcp

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: replace Yelp-specific hostnames with generic examples

Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)

This makes the code more generic and suitable for open source.
All 13 tests still passing.

* remove accidentally added files

* fix formatting

* add new test for get_base_url

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-29 19:11:32 -07:00
Timothée Lecomte
eb0e4f34dc
docs: use custom-llm-provider header in examples (#16055) 2025-10-29 19:06:48 -07:00