Commit Graph

23973 Commits

Author SHA1 Message Date
Mattias Andersson
89f71af4cd Add possibility to configure resources for migrations-job in Helm chart 2025-08-14 17:08:26 +02:00
Krrish Dholakia
0288ed35da test: update tests 2025-08-13 23:33:32 -07:00
Krrish Dholakia
b53962dee2 test: update test 2025-08-13 23:09:18 -07:00
Ishaan Jaff
cd893134b7 test team endpoints 2025-08-13 18:43:50 -07:00
Ishaan Jaff
ce4210a17a bump litellm proxy extras 2025-08-13 17:50:10 -07:00
Ishaan Jaff
086621e3d3 test_handle_jwt.py 2025-08-13 17:27:50 -07:00
Ishaan Jaff
8e76f8e7d0
[Feat] Team Member Rate Limits + Support for using with JWT Auth (#13601)
* fix - assign tpm/rpm limit onJWT

* add team member rpm/tpm limits

* update - rate limiter v3 with team member rate limits

* update utils

* fixes for LiteLLM_BudgetTable

* undo change

* add TeamMemberBudgetHandler

* add _process_team_member_budget_data

* add get_team_membership

* add safe_get_team_member_rpm_limit and safe_get_team_member_tpm_limit

* LiteLLM_TeamMembership

* add LiteLLM_TeamMembership rate limit for JWTs

* fix

* tests
2025-08-13 17:21:36 -07:00
Ishaan Jaff
76d25926d4
[Feat] New provider - Azure AI Flux Image Generation (#13592)
* init files

* add AzureFoundryModelInfo

* fix api_version property

* add azure_ai img gen

* use AzureFoundryModelInfo

* get_base_image_generation_call_args

* add azure_ai/FLUX-1.1-pro

* add util for route_image_generation_cost_calculator

* docs azure ai flux

* fixes for flux

* fixes for AzureFoundryFluxImageGenerationConfig

* ruff fix
2025-08-13 17:20:30 -07:00
Ishaan Jaff
fb325cbb5e
fix Build from litellm pip package (#13603) 2025-08-13 17:19:43 -07:00
Krrish Dholakia
2e21f1d07a fix(router.py): fix test 2025-08-13 16:27:13 -07:00
Ishaan Jaff
75bcfbb76a
[Feat] New model vertex_ai/deepseek-ai/deepseek-r1-0528-maas (#13594)
* add ertex_ai/deepseek-ai/deepseek-r1-0528-maas

* fix init

* test_model_info_for_vertex_ai_deepseek_model
2025-08-13 13:44:45 -07:00
Krish Dholakia
ffd165c9b7
Merge pull request #13549 from BerriAI/litellm_mcp_config_fix
[fix] Enhance MCPServerManager with access groups and description support
2025-08-13 10:51:19 -07:00
Krish Dholakia
a1f6ade9f6
Merge pull request #13582 from BerriAI/remove-network-response-error
Remove ambiguous network response error
2025-08-13 10:50:05 -07:00
Michael Verunica
37e57a0e5f
fix(azure): remove trailing semicolon in Content-Type header for image generation (#13584) 2025-08-13 09:43:02 -07:00
Krish Dholakia
5e45fcdff0
Merge pull request #13375 from colesmcintosh/fix/ollama-gpt-oss-thinking-field 2025-08-13 07:58:57 -07:00
Krzysztof Gąsiorowski
0b28930ad4
[Fix] Hide sensitive data in /model/info - azure entra client_secret (#13577)
* Remove litellm_params.client_secret from /model/info

Added pop of client_secret (Azure provider secret) from litellm_params in remove_sensitive_info_from_deployment used by /model/info endpoints

* Added test for litellm.proxy.common_utils.openai_endpoint_utils.remove_sensitive_info_from_deployment
2025-08-13 07:35:53 -07:00
tanjiro
be109c2180 put the error toast on the ui 2025-08-13 23:05:11 +09:00
tanjiro
06ee35f74c replace text error with json error 2025-08-13 22:32:57 +09:00
Krrish Dholakia
5ae44e3275 fix(router.py): fix cooldown increment logic 2025-08-12 23:33:56 -07:00
Krish Dholakia
6afaf5721a
[Fix] Streaming - consistent 'finish_reason' chunk index (#13560)
* feat(model_response_utils.py): new function to check if modelresponsestream is empty

used for checking https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): skip chunk if empty

Fixes https://github.com/BerriAI/litellm/issues/13348

* fix(streaming_handler.py): add is_empty logic to async flow
2025-08-12 23:21:57 -07:00
Krish Dholakia
f00e891004
LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555)
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params

Fixes issue where 'user' was not being sent in request to litellm proxy via sdk

* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default

* docs: cleanup docs

* fix(proxy_cli.py): remove --use_prisma_migrate flag

* refactor: remove references to use_prisma_migrate env var

This is now the default flow for db migrations
2025-08-12 22:03:39 -07:00
Ishaan Jaff
38d0736599 bump fastapi-offline 2025-08-12 21:19:29 -07:00
Sameer Kankute
74496eefea
Implement GitHub Action to auto-label issues with provider keywords (#13537)
* Add auto issue labelling for llm_translation

* Add logging

* make the formatting better

* make the formatting better

* make the formatting better

* make the formatting better
2025-08-12 21:18:20 -07:00
Jugal D. Bhatt
e51c15705c
[Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel (#13550)
* [Feat] Add CredentialDeleteModal component and integrate with CredentialsPanel

* Implement CredentialDeleteModal for confirming credential deletions.
* Update CredentialsPanel to manage delete modal state and handle credential deletion.
* Ensure user confirmation before deleting credentials to prevent accidental removals.

* Update CredentialDeleteModal to use smart quotes for credential name in confirmation message

* Enhance CredentialDeleteModal with confirmation input and validation

* Added input field for user to confirm credential deletion by typing the credential name.
* Updated modal behavior to handle confirmation and cancellation actions.
* Removed unnecessary state management in CredentialsPanel for delete modal visibility.

* Fix HTML entity for apostrophe in CredentialDeleteModal confirmation message
2025-08-12 21:17:39 -07:00
Ishaan Jaff
1249385a99
[Feat] GEMINI CLI - Add Token Counter for VertexAI Models (#13558)
* add VertexAIModelInfo

* working API call to vertex ai

* add count_tokens MODE

* _construct_url

* test_vertex_ai_gemini_token_counting_with_contents
2025-08-12 20:53:47 -07:00
Ishaan Jaff
74ddd21067
[Bug fix] SCIM Team Memberships - handle metadata (#13553)
* test_update_group_metadata_serialization_issue

* fixes for update_group

* test_update_group_metadata_serialization_issue
2025-08-12 20:53:12 -07:00
tanjiro
ebae03cf93
Enhance chart readability with short-form notation for large numbers (#12370)
* format y-axis value for total tokens

* format y-axis for the rest of the charts on model-activity

* revert changes for requests per day

* labels modified to plain text

* added plain text label for api_requests and spend

* minor

* move components to utils
2025-08-12 17:24:03 -07:00
tanjiro
88e4d302a2
Fix internal users table overflow (#12736)
* modify column name

* fix overflow

* remove height
2025-08-12 17:23:23 -07:00
tanjiro
e4364ad1c4
Team Member Permissions Page - Access Column Changes (#13145)
* revert prettier

* move allow access to right
2025-08-12 17:22:24 -07:00
tanjiro
911b0cfd73
Display Error from Backend on the UI - Keys Page (#13435)
* fix sso logout

- add a new login page with sso button

* lint fix

* lint fix

* lint fix

* fix tests

* fix test

* Revert "fix test"

This reverts commit 74eb7345710892d5a9d02baec0ef389b98d0dde3.

* Reapply "fix test"

This reverts commit 72d0b2d4c62f6bb9351a7656ff88efc2ba91aef7.

* add host to add modal

* close modal after save is clicked. and auto-refresh

* show old values in edit modal

* send the whole payload on edit

* Update settings.tsx

* resolve conflict

* fix conflict

* merge main

* first draft of notifications added to settings

* add error compatibility by taking errors from the backend

- db errors
- auth errors

* add support for different types of errors

* minor

* name change

* email alerts page notifications modified

* remove unused code

* move create_key to organisms/ folder

* move view_key_table to templates

* keys page notifications

* move regenerate_key to organisms

* notifications for regenerate key modal

* move key info view and key edit view to templates/ folder

* replace "Network response was not ok" with error data

* resolve merge conflict
2025-08-12 17:21:04 -07:00
Krrish Dholakia
280ad6f049 fix(main.py): add sambanova api base support to embeddings 2025-08-12 17:17:19 -07:00
Jorge Piedrahita Ortiz
6a77780c0e
Feat/sambanova embeddings (#13308)
* add sambanova embeddings

* fmt

* minor fix

* add sambanova embeddings call

* fmt

* include embeddings in sambanova docs

* add sambanova embeddigns unit test

* remove unused import

* minor fix

* fmt

* update sambanova embeding to inherit from BaseEmbeddingConfig

* fmt

* fmt

* hot fix

* fmt
2025-08-12 17:15:26 -07:00
Jugal Bhatt
bb32ec991f Merge branch 'main' into litellm_mcp_config_fix 2025-08-12 17:10:18 -07:00
Ishaan Jaff
afe159bb8b
[Feat] GEMINI CLI Integration - Add /countTokens endpoint support (#13545)
* stash changes for token counter

* working TokenCountRequest

* working acount_tokens

* add GoogleAIStudioTokenCounter

* re-use validate_environment

* fixes count_tokens

* fixes google_count_tokens

* fixes token counter base class

* fix TokenCountResponse

* fix - use BaseTokenCounter

* add should_use_token_counting_api

* fixes for GoogleAIStudioTokenCounter

* fixes for should_use_token_counting_api

* fixes for google_count_tokens

* fixes for /messages count_tokens

* fixes for should_use_token_counting_api

* working e2e gemini token counter

* ruff check fixes

* fixes for token counter

* fixes for TokenCountResponse

* cleanup TokenCountRequest

* add TokenCountDetailsResponse

* fix use well typed Responses

* fix typing for TokenCountDetailsResponse

* test_vertex_ai_gemini_token_counting_with_contents

* fixes for TokenCountDetailsResponse

* test fixes

* test_factory_registration

* test_proxy_token_counter.py

* TestGoogleAIStudioTokenCounter

* fix token_counter
2025-08-12 16:19:58 -07:00
Jugal Bhatt
f487816b9f [fix] Enhance MCPServerManager with access groups and description support
* Added access_groups and description fields to MCPServerManager for better server configuration.
* Implemented tests to verify integration of config-based servers with database servers, ensuring correct handling of access_groups and description.
* Updated add_update_server method to accommodate new fields and validate server addition in the registry.
2025-08-12 14:22:40 -07:00
Ishaan Jaff
828e3e3deb
[Feat] Add Streaming support + Docs for bedrock gpt-oss model family (#13346)
* add openai.gpt-oss-20b-1:0

* update BEDROCK_CONVERSE_MODELS

* openai.gpt-oss-20b-1:0 fixes

* fix PDF input

* fix for should_fake_stream

* TestBedrockGPTOSS

* should_fake_stream

* update supports vision field for openai.gpt-oss models

* fixes for bedrock gpt oss

* fixes for should_fake_stream

* docs bedrock gpt oss models
2025-08-12 08:39:36 -07:00
Ishaan Jaff
008ea864a7
[Feat] - Add key/team logging for Langfuse OTEL Logger (#13512)
* feat - add key/team logging for LF

* test_construct_dynamic_otel_headers_with_langfuse_keys

* update LangfuseOtelLogger

* test_construct_dynamic_otel_headers_with_langfuse_keys

* cleanup

* OpenTelemetryConfig fixes

* fix code qa checks

* TestLangfuseOtelIntegration
2025-08-11 22:06:25 -07:00
tanjiro
d5135bba31
Allow unsetting TPM and RPM - Teams Settings (#13430)
* modified tpm rpm for 0 and null values

* show unlimited if tpm/rpm 0 or null

* bring back unlimited
2025-08-11 18:11:43 -07:00
NULL
f3dcae2528
feat: add CometAPI provider support with chat completions and streaming (#13458)
* feat: add CometAPI support with config, error handling and tests

* fix: specify type for extra_body in CometAPIConfig

---------

Signed-off-by: NULL <129579691+TensorNull@users.noreply.github.com>
2025-08-11 18:06:37 -07:00
breno-aumo
7484a19edf
Fix OCI streaming (#13437)
* add oci models to model_prices_and_context_window.json

* remove unsupported and unavailable oci models from docs

* fix: split data chunks when using async streaming

* add oci models to model_prices_and_context_window.json

* fix: split data chunks when using async streaming

* revert model.json modifications
2025-08-11 18:04:58 -07:00
Ishaan Jaff
49ca4dc792
Sending tags to LiteLLM Proxy (#13517) 2025-08-11 17:58:50 -07:00
Krrish Dholakia
21e067e47f fix: fix check 2025-08-11 17:56:19 -07:00
Krrish Dholakia
92ebf5b918 fix(router.py): fix print statement 2025-08-11 17:46:14 -07:00
Jugal D. Bhatt
5efe59e6e2
[Draft] [LLM Translation] Add model id check (#13507)
* Add model id check

* add ruff fix
2025-08-11 15:41:08 -07:00
Ishaan Jaff
9f78287000
[Bug Fix]: Azure OpenAI GPT-5 max_tokens + reasoning param support (#13510)
* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* add AzureOpenAIGPT5Config

* test_azure_gpt5_supports_reasoning_effort

* test_azure_gpt5_reasoning

* test_azure_gpt5_reasoning

* ruff check fixes

* docs azure gpt5
2025-08-11 15:40:53 -07:00
Cole McIntosh
8197fd74d5 Revert "merge from upstream"
This reverts commit e6ca91869a.
2025-08-11 07:54:47 -06:00
Cole McIntosh
e6ca91869a merge from upstream 2025-08-11 07:38:16 -06:00
Cole McIntosh
d5d7e00d34 Enhance chunk parsing for Ollama streaming responses
Updated the chunk_parser method to return a ModelResponseStream when handling 'thinking' field content, allowing UIs to render reasoning information. Adjusted tests to verify the new behavior, ensuring that reasoning content is correctly returned in the response.
2025-08-11 07:02:25 -06:00
Cole McIntosh
7a1c6efa0a
Merge branch 'BerriAI:main' into fix/ollama-gpt-oss-thinking-field 2025-08-11 06:54:51 -06:00
Ishaan Jaff
1cd827874f
[Bug Fix] - Allow using reasoning_effort for gpt-5 model family and reasoning for Responses API (#13475)
* test_openai_gpt5_reasoning

* test_openai_gpt5_reasoning_effort_parameter

* add OpenAIGPT5ResponsesAPIConfig

* test_openai_gpt5_reasoning_effort_parameter

* fixes
2025-08-10 09:55:36 -07:00