ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
34,537 Commits 9 Branches 7 Tags 935 MiB
6034efb9c6
Commit Graph

1398 Commits

Author SHA1 Message Date
Sameer Kankute
6034efb9c6 Add flex pricing 2026-03-07 01:02:18 +05:30
Sameer Kankute
7d524fe5a8 Fix model map 2026-03-06 22:49:37 +05:30
Sameer Kankute
dd7ab3f01f Fix model map 2026-03-06 22:46:32 +05:30
Sameer Kankute
a7967801b0 Fixed o file 2026-03-06 22:29:25 +05:30
Sameer Kankute
d8f139fe4d feat(openai): add 272K tier pricing for GPT-5.4/5.4-pro 
Prompts >272K input tokens priced at 2x input, 1.5x output for full session
(standard, batch, flex). Applies to models with 1.05M context window (gpt-5.4,
gpt-5.4-pro).

- Add input/output_cost_per_token_above_272k_tokens to model_prices
- Add above_272k fields to ModelInfoBase and get_model_info extraction
- Add test_generic_cost_per_token_gpt54_above_272k_tokens

Made-with: Cursor
 2026-03-06 22:26:14 +05:30
Sameer Kankute
6ba2e9f10f feat(gpt-5): add supports_none_reasoning_effort and supports_xhigh_reasoning_effort to model cost map 
- Shift from hardcoded model checks to dynamic lookup via _supports_factory
- Add supports_none_reasoning_effort for gpt-5.1/5.2/5.4 chat variants
- Add supports_xhigh_reasoning_effort for gpt-5.1-codex-max, gpt-5.2, gpt-5.4+
- Update model_prices_and_context_window.json and backup
- Add ProviderSpecificModelInfo types for new fields
- Fix Azure: use _supports_reasoning_effort_level instead of removed is_model_gpt_5_1_model

Made-with: Cursor
 2026-03-06 18:15:32 +05:30
Daniel Yudelevich
8b2dd4f65a
fix(pricing): add supports_web_search for OpenAI gpt-5.1/5.2/5.3 models (#22934) 
OpenAI gpt-5.1, gpt-5.2, and gpt-5.3 chat models all support the
`web_search_options` parameter, but the model cost registry was missing
the `supports_web_search` flag. Only `gpt-5.2-pro` had it set.

Models updated:
- gpt-5.1, gpt-5.1-2025-11-13, gpt-5.1-chat-latest
- gpt-5.2, gpt-5.2-2025-12-11, gpt-5.2-chat-latest
- gpt-5.3-chat-latest
 2026-03-05 20:38:27 -08:00
Sameer Kankute
050503469a feat(openai): add gpt-5.4-pro to model map 
- Add gpt-5.4-pro and gpt-5.4-pro-2026-03-05 snapshot
- Input: $30/1M tokens, Output: $180/1M tokens
- 1.05M context window, 128K max output tokens
- Priority pricing for >272K input tokens (2x input, 1.5x output)
- Supports reasoning.effort: medium, high, xhigh
- Responses API, Chat Completions, Batch endpoints

Made-with: Cursor
 2026-03-06 09:52:56 +05:30
Sameer Kankute
20ec949cf1
Merge pull request #22734 from vincentkoc/vincentkoc-code/chatgpt-53-oauth-models 
feat(models): add ChatGPT 5.3/5.4 aliases + OpenAI gpt-5.4-pro
 2026-03-06 08:59:12 +05:30
Antti Puurula
20ce6ddbef
Fix apac.anthropic.claude-sonnet-4-6 -> au.anthropic.claude-sonnet-4-6 (#21466) 2026-03-05 15:54:26 -08:00
Vincent Koc
f1e4e4a0ce feat(models): add gpt-5.4-pro and chatgpt 5.4 aliases 2026-03-05 16:50:38 -05:00
Vincent Koc
32e5e55f84 feat(models): add chatgpt gpt-5.3 oauth aliases 2026-03-05 16:50:38 -05:00
Sameer Kankute
8dca085640
Merge pull request #22916 from BerriAI/litellm_gpt-5.4_day_0 
Add day 0 support for gpt-5.4
 2026-03-05 23:41:35 +05:30
Sameer Kankute
3b457b5d8e Add day 0 support for gpt-5.4 2026-03-05 23:40:24 +05:30
Sameer Kankute
b9a8d42882 Add day 0 support for gpt-5.4 2026-03-05 23:26:24 +05:30
Sameer Kankute
5183a6e850
Merge pull request #22866 from mubashir1osmani/feat/bedrock-mantle-provider-clean 
feat: bedrock mantle provider
 2026-03-05 18:24:00 +05:30
Sameer Kankute
bb1297fe1b feat(cost): add azure_ai/mistral-document-ai-2512 to model cost map 
Made-with: Cursor
 2026-03-05 16:07:46 +05:30
mubashir1osmani
df7e3aa1e5 feat(provider): add Amazon Bedrock Mantle as a first-class provider 
Adds `bedrock_mantle` provider for Amazon Bedrock's OpenAI-compatible
inference engine (Project Mantle). Previously users had to use this as
a generic openai_compatible provider, which resulted in incorrect pricing
(OpenAI rates instead of Bedrock rates).

Changes:
- New `BedrockMantleChatConfig` extending `OpenAILikeChatConfig`
  - Regional API base: `https://bedrock-mantle.{region}.api.aws/v1`
  - Auth via `BEDROCK_MANTLE_API_KEY` env var
  - Region resolution: BEDROCK_MANTLE_REGION > AWS_REGION > us-east-1
  - Supports reasoning for gpt-oss models
- Added `BEDROCK_MANTLE` to `LlmProviders` enum
- Added 4 models with correct AWS Bedrock pricing to both pricing files:
  - bedrock_mantle/openai.gpt-oss-120b ($0.15/M in, $0.60/M out)
  - bedrock_mantle/openai.gpt-oss-20b ($0.075/M in, $0.30/M out)
  - bedrock_mantle/openai.gpt-oss-safeguard-120b
  - bedrock_mantle/openai.gpt-oss-safeguard-20b
- Wired provider into get_llm_provider_logic, get_supported_openai_params,
  main.py routing, utils.py map_openai_params + ProviderConfigManager,
  and _lazy_imports_registry
- 19 unit tests covering registration, config, provider resolution, pricing

Usage:
  os.environ["BEDROCK_MANTLE_API_KEY"] = "your-key"
  litellm.completion(model="bedrock_mantle/openai.gpt-oss-120b", ...)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-05 00:03:40 -05:00
Marty Sullivan
0909eee744
add missing bedrock models (#22810) 2026-03-04 15:13:09 -08:00
Chesars
0e1a633e30 fix: update mode to realtime for gemini-live models 
The mode field is used by health checks to determine the correct
check method (WebSocket for realtime vs REST for chat).
 2026-03-04 19:43:23 -03:00
Chesars
ddf9598f30 fix: use /v1/realtime for gemini/ provider live model 
The gemini/ prefix indicates Google AI Studio, which uses /v1/realtime
endpoint (OpenAI-compatible), not /vertex_ai/live.
 2026-03-04 19:43:23 -03:00
Chesars
20a41a67d6 fix: update gemini-live model supported_endpoints to /vertex_ai/live 
The gemini-live-2.5-flash-preview-native-audio-09-2025 model only works
with WebSocket (Live API), not REST endpoints. Changed supported_endpoints
from /v1/chat/completions to /vertex_ai/live to reflect the actual
passthrough endpoint available in LiteLLM proxy.
 2026-03-04 19:43:23 -03:00
Peter Dave Hello
007bea10b8
Add Support for OpenAI's Chat-GPT 5.3 Chat model (#22693) 
Reference:
- https://openai.com/index/gpt-5-3-instant/
- https://developers.openai.com/api/docs/models/gpt-5.3-chat-latest
 2026-03-03 20:27:05 -08:00
Sameer Kankute
7a83acf086
Merge pull request #22620 from OiPunk/codex/litellm-22619-deepinfra-function-calling 
fix: add missing supports_function_calling for deepinfra models
 2026-03-04 08:51:21 +05:30
Varad Khonde
7d65df351f feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model 2026-03-03 23:40:41 +05:30
Sameer Kankute
daa0397f30
Merge pull request #22674 from BerriAI/litellm_gemini-3.1-flash-lite-preview 
Day 0 gemini 3.1 flash lite preview support
 2026-03-03 22:23:17 +05:30
Sameer Kankute
9d06106af0 Fix gemini-3.1-flash-lite-preview for streaming 2026-03-03 22:22:57 +05:30
Sameer Kankute
c3fe4634b6 Add correct pricing for gemini 3.1 flash lite 2026-03-03 22:06:16 +05:30
Cesar Garcia
a0cc550f48
Merge pull request #22665 from Chesars/feat/add-zai-glm-5-model-pricing 
feat(models): add zai/glm-5 and zai/glm-5-code to model cost map
 2026-03-03 12:17:42 -03:00
Chesars
75518c3ca7 feat(models): add zai/glm-5 and zai/glm-5-code to model cost map 
Add native ZhipuAI GLM-5 and GLM-5-Code model entries with pricing
from docs.z.ai/guides/overview/pricing.
 2026-03-03 12:03:40 -03:00
Cesar Garcia
dfd7203d7f
Merge pull request #22650 from Chesars/fix/add-missing-dashscope-glm5-pricing 
fix(pricing): add 7 missing DashScope model pricing entries
 2026-03-03 11:53:30 -03:00
Cesar Garcia
105a40ba4f
Merge pull request #22649 from Chesars/fix/add-missing-openrouter-model-pricing 
fix(pricing): add 5 missing OpenRouter model pricing entries
 2026-03-03 11:53:07 -03:00
Sameer Kankute
24ec7f882f
Revert "feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model" 2026-03-03 19:35:26 +05:30
Varad Khonde
d66f8bc15d feat(togetherai): add support for togetherai/Qwen3.5-397B-A17B model 2026-03-03 19:17:28 +05:30
Chesars
7977126181 fix(pricing): add 7 missing DashScope model pricing entries 
Fixes #22646

Adds pricing for DashScope models that were missing from the cost map,
causing $0 spend tracking in the proxy dashboard:

- dashscope/qwen3-max-2026-01-23 (tiered, same as qwen3-max)
- dashscope/qwen3-next-80b-a3b-instruct ($0.15/$1.20 per 1M)
- dashscope/qwen3-next-80b-a3b-thinking ($0.15/$1.20 per 1M)
- dashscope/qwen3-vl-235b-a22b-instruct ($0.40/$1.60 per 1M)
- dashscope/qwen3-vl-235b-a22b-thinking ($0.40/$4.00 per 1M)
- dashscope/qwen3-vl-32b-instruct ($0.16/$0.64 per 1M)
- dashscope/qwen3-vl-32b-thinking ($0.16/$2.87 per 1M)
 2026-03-03 09:52:17 -03:00
Chesars
409208771e fix(pricing): add 5 missing OpenRouter model pricing entries 
Fixes #22609

Adds pricing for OpenRouter models that were routing correctly but
returning $0 for spend tracking due to missing cost map entries:

- openrouter/anthropic/claude-sonnet-4.6 ($3.00/$15.00 per 1M tokens)
- openrouter/google/gemini-3.1-pro-preview ($2.00/$12.00 per 1M tokens)
- openrouter/openai/gpt-5.1-codex-max ($1.25/$10.00 per 1M tokens)
- openrouter/qwen/qwen3-coder-plus ($1.00/$5.00 per 1M tokens)
- openrouter/z-ai/glm-5 ($0.80/$2.56 per 1M tokens)
 2026-03-03 09:24:49 -03:00
Sameer Kankute
851be58775 Add day 0 support of gemini-3.1-flash-lite-preview 2026-03-03 15:07:47 +05:30
liweiguang
81ddf08494 fix: add missing supports_function_calling for deepinfra models 
All 55 deepinfra models that had `supports_tool_choice: true` were
missing the `supports_function_calling` flag, causing
`litellm.supports_function_calling()` to incorrectly return False.

Fixes #22619

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-03 12:12:43 +08:00
Ishaan Jaff
86d5b4c632
feat: add Nebius AI Studio models to model_prices_and_context_window.json (#22614) 
Add 30 Nebius AI Studio models covering:
- Text-to-text: DeepSeek (R1, R1-0528, R1-Distill, V3, V3-0324), Meta Llama
  (3.1-8B/70B/405B, 3.3-70B), Qwen (3-235B/32B/30B/14B/4B, 2.5-72B/32B,
  2.5-Coder-7B, QwQ-32B), Mistral Nemo, NousResearch Hermes-3, NVIDIA
  Nemotron Ultra/Super, Google Gemma-3-27B, Llama-Guard-3
- Vision: Qwen2.5-VL-72B, Qwen2-VL-72B, Qwen2-VL-7B
- Embedding: BAAI/bge-en-icl, BAAI/bge-multilingual-gemma2, intfloat/e5-mistral-7b

Pricing sourced from https://nebius.com/prices-ai-studio (base flavor).
Context windows sourced from https://docs.nebius.com/studio/inference/models/

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
 2026-03-02 18:43:07 -08:00
Ishaan Jaff
bfceb7fc3f
feat(perplexity): add embedding support for pplx-embed-v1 models (#22610) 
* feat: add Perplexity embedding support (pplx-embed-v1)

Add support for Perplexity AI's embedding models via the LLM HTTP handler:

Models:
- pplx-embed-v1-0.6b (1024 dims, 32K context, $0.004/1M tokens)
- pplx-embed-v1-4b (2560 dims, 32K context, $0.03/1M tokens)

Implementation:
- PerplexityEmbeddingConfig in litellm/llms/perplexity/embedding/
- Registered in ProviderConfigManager, __init__.py lazy imports, main.py dispatch
- Model pricing added to model_prices_and_context_window.json
- Supports dimensions and encoding_format parameters
- Uses base_llm_http_handler.embedding() pattern

Tests:
- 19 unit tests covering transformation, params, URLs, provider config, model info

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* docs: add Perplexity AI embeddings documentation

- Create providers/perplexity_embedding.md with SDK and proxy usage examples
- Convert Perplexity from flat doc to category in sidebars.js
- Category includes existing chat/responses doc + new embeddings doc
- Covers pplx-embed-v1-0.6b and pplx-embed-v1-4b models
- Documents supported parameters (dimensions, encoding_format)
- Includes proxy config and curl examples

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: decode Perplexity base64_int8 embeddings to OpenAI-format float arrays

Perplexity returns embeddings as base64-encoded signed int8 values by default,
not float arrays like OpenAI. This commit adds decoding in
transform_embedding_response so the proxy returns standard OpenAI-compatible
float arrays (normalized to [-1, 1]).

- Added _decode_base64_embedding() static method
- Handles both base64 strings (decoded) and float lists (passthrough)
- Added 3 new tests for base64 decoding + passthrough

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
 2026-03-02 17:37:50 -08:00
Cesar Garcia
835a2c3dc6
Merge pull request #22583 from Chesars/fix/add-bedrock-cache-token-pricing 
fix(pricing): add missing cache token pricing for 24 Bedrock Claude models
 2026-03-02 18:46:28 -03:00
Cesar Garcia
680b9ee9f2
Merge pull request #22586 from Chesars/fix/update-gemini-deprecation-dates 
fix: update Gemini model deprecation dates
 2026-03-02 18:45:30 -03:00
Cesar Garcia
a54a1d27d7
Merge pull request #22596 from Chesars/fix/add-dashscope-models-pricing 
fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus
 2026-03-02 18:45:08 -03:00
Cesar Garcia
229eb5234d
Merge pull request #22601 from Chesars/fix/update-mistral-models-pricing 
feat: add missing Mistral models and update pricing
 2026-03-02 18:44:38 -03:00
Chesars
884f7c5e4e fix: update mistral-small-latest to match Small 3.2 specs 
mistral-small-latest now points to Small 3.2 (since June 2025).
Updated pricing from $0.10/$0.30 to $0.06/$0.18 per 1M tokens,
context from 32k to 131k, and added vision support to match
mistral-small-3-2-2506.
 2026-03-02 18:36:05 -03:00
Chesars
abb7eb250a fix: remove retired Saba model from new entries 
Saba was retired on 9/30/2025 per Mistral docs, replaced by Small 3.2.
 2026-03-02 18:19:46 -03:00
Chesars
bd822a7a68 fix: add supports_response_schema to Ministral 3 models 
Ministral 3 (3B, 8B, 14B) support structured outputs per Mistral docs.
 2026-03-02 18:19:02 -03:00
Chesars
619f53d55a feat: add missing Mistral models and update outdated pricing 
Add 9 new Mistral models (mistral-large-2512, mistral-medium-3-1-2508,
mistral-small-3-2-2506, ministral-3-3b/8b/14b-2512, saba-2502,
magistral-medium/small-1-2-2509) and update mistral-large-latest,
mistral-large-3, and mistral-medium-latest with correct pricing and
context windows.

Fixes #22585
 2026-03-02 18:02:41 -03:00
Chesars
f0e571413d fix: add missing pricing for dashscope/qwen3.5-plus and dashscope/qwen3-vl-plus 
Fixes #22591 - These models were missing from the pricing JSON, causing
$0 cost tracking when routed via the dashscope/* wildcard.

Pricing sourced from official Alibaba Cloud Model Studio docs (international tier).
 2026-03-02 17:24:19 -03:00
Chesars
ee3475d187 fix: correct gemini/gemini-2.0-flash-lite-preview-02-05 deprecation_date 
Update from 2025-12-02 to 2025-12-09 per
https://ai.google.dev/gemini-api/docs/deprecations
 2026-03-02 15:54:24 -03:00