Commit Graph

33292 Commits

Author SHA1 Message Date
Ryan Crabbe
e0ddb2a525 fix: guard print_aggregate against empty latencies 2026-02-23 10:06:10 -08:00
Ryan Crabbe
fb123ec52b refactor: use method-level import for MockOpenAITransport 2026-02-23 09:58:46 -08:00
Ryan Crabbe
95d9514054 fix: add auth headers and empty latencies guard to benchmark script 2026-02-23 09:58:40 -08:00
Ryan Crabbe
8244ad1f0e docs: tweak benchmarks wording 2026-02-23 09:50:02 -08:00
Ryan Crabbe
5b41b009f6 docs: add network mock benchmarking section 2026-02-23 09:44:02 -08:00
Ryan Crabbe
d99d87f614 clean up mock transport: remove streaming, add defensive parsing 2026-02-23 09:16:47 -08:00
Ryan Crabbe
94b76ea9ad feat: add network_mock transport for benchmarking proxy overhead without real API calls
Intercepts at httpx transport layer so the full proxy path (auth, routing,
OpenAI SDK, response transformation) is exercised with zero-latency responses.
Activated via `litellm_settings: { network_mock: true }` in proxy config.
2026-02-21 17:52:39 -08:00
Ishaan Jaffer
356eb5a413 docs fix 2026-02-21 17:51:45 -08:00
Ishaan Jaffer
522954fe0d docs fix 2026-02-21 17:47:44 -08:00
Ishaan Jaffer
45bef9ade8 docs fix 2026-02-21 17:46:01 -08:00
Ishaan Jaffer
5e71f6128b docs fix 2026-02-21 17:40:39 -08:00
Ishaan Jaffer
e157f5a8f2 docs fix 2026-02-21 17:35:16 -08:00
Ishaan Jaffer
661c6faac6 docs fix 2026-02-21 17:28:04 -08:00
Ishaan Jaffer
efebd37183 docs fix 2026-02-21 17:28:04 -08:00
yuneng-jiang
6228148963
Merge pull request #21857 from BerriAI/litellm_yj_docs_feb21
[Doc] Server Root Path Regression Incident Doc#21853
2026-02-21 17:26:43 -08:00
yuneng-jiang
386a8fbd5b
Merge branch 'main' into litellm_yj_docs_feb21 2026-02-21 17:21:35 -08:00
Ishaan Jaffer
7e567a5352 ui new build 2026-02-21 17:16:10 -08:00
Ishaan Jaffer
67a64266ef fix rendering 2026-02-21 17:15:18 -08:00
yuneng-jiang
823bb023df
Merge branch 'main' into litellm_yj_docs_feb21 2026-02-21 17:12:28 -08:00
Ishaan Jaffer
44f69038de UI new build 2026-02-21 17:00:45 -08:00
Ishaan Jaffer
52294029a0 test_vertex_ai_gemini_2_5_pro_streaming 2026-02-21 16:59:22 -08:00
Ishaan Jaffer
2270a3aaf3 Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth"
This reverts commit 0e1db3f7e4, reversing
changes made to 7e2d6f2355.
2026-02-21 16:57:42 -08:00
milan-berri
5390469c1b
fix: replace Zapier webhook with Google Form for survey submission (#21621)
* Replace Zapier webhook with Google Form for survey submission

* Add back error logging for survey submission debugging

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2026-02-21 16:55:59 -08:00
Ishaan Jaffer
ab032c292c docs fix 2026-02-21 16:36:22 -08:00
ryan-crabbe
0e1db3f7e4
Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth
perf: optimize user_api_key_auth
2026-02-21 16:23:54 -08:00
ryan-crabbe
ce22524ac5
Merge branch 'main' into litellm_perf_user_api_key_auth 2026-02-21 16:05:33 -08:00
yuneng-jiang
241141d70b Merge branch 'doc_yj_feb21' of github.com:BerriAI/litellm into doc_yj_feb21 2026-02-21 16:05:27 -08:00
yuneng-jiang
aefc7c14f6 Merge remote-tracking branch 'origin' into doc_yj_feb21 2026-02-21 16:05:07 -08:00
Ishaan Jaffer
7e2d6f2355 ui fix 2026-02-21 16:05:03 -08:00
yuneng-jiang
70fd2aa219 fixing syntax 2026-02-21 16:04:42 -08:00
Ryan Crabbe
643c9b6c04 Merge remote-tracking branch 'origin/main' into litellm_perf_user_api_key_auth 2026-02-21 16:03:31 -08:00
yuneng-jiang
2e8d942d17
Merge branch 'main' into doc_yj_feb21 2026-02-21 16:00:46 -08:00
yuneng-jiang
153bf1d856 server root path regression doc 2026-02-21 15:57:06 -08:00
Ishaan Jaffer
d31d5b8486 fix failing tests 2026-02-21 15:48:26 -08:00
Ishaan Jaffer
775fb79260 fix 2026-02-21 15:45:03 -08:00
Ishaan Jaffer
26ea29afd3 test_get_usage_as_dict 2026-02-21 15:39:06 -08:00
Ishaan Jaffer
a58232ac5b fix prisma 2026-02-21 15:35:50 -08:00
Krish Dholakia
1f7eeb274c
Agent Builder - improve rejected response detection based on agent response (#21850)
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent

* feat: new response rejection detector

* fix: multiple fixes

* feat: add mcp tools support to agent builder

create an agent with access to llm's + mcp servers
2026-02-21 15:34:42 -08:00
Krish Dholakia
9fc6fd647c
Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817)
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent
2026-02-21 15:32:47 -08:00
Ishaan Jaffer
2acc93e451 BUMP 2026-02-21 15:28:18 -08:00
Ishaan Jaff
bab4127cae
fix(tests): fix flaky test_use_prisma_db_push_flag_behavior (#21849)
Replace Click CliRunner with standalone_mode=False to avoid
"I/O operation on closed file" errors caused by Click's stream
isolation in CI environments.
2026-02-21 15:23:55 -08:00
Ishaan Jaff
f74a1c94df
test(router): add coverage tests for _is_complexity_router_deployment and init_complexity_router_deployment (#21848) 2026-02-21 15:21:10 -08:00
Ishaan Jaff
6dc9823926
docs(release-notes): update v1.81.14 - split guardrail sections, add eval results, fix key highlights and section placement (#21847) 2026-02-21 15:18:46 -08:00
Ryan Crabbe
c7ad8053b1 Merge origin/main into litellm_perf_user_api_key_auth
Resolve conflicts:
- pass_through_endpoints.py: take main's version, re-apply
  MAPPED_PASS_THROUGH_PREFIXES startswith(tuple) optimization
- test_user_api_key_auth.py: keep both auth optimization regression
  tests and JWT admin identity field tests
2026-02-21 15:14:20 -08:00
Ishaan Jaff
eac3ae8121
docs: update v1.81.14 release notes - guardrail model garden, complexity router placement (#21843)
* docs(release-notes): update v1.81.14 key highlights and section placement

* docs(release-notes): rewrite key highlights and add guardrail narrative section

* docs(release-notes): rewrite guardrail narrative to match release notes style

* docs(release-notes): add guardrail eval results section
2026-02-21 15:10:21 -08:00
Ishaan Jaff
9f459c5c57
fix(logging): preserve pass-through endpoint response_cost (#21844)
* fix(logging): preserve pass-through endpoint response_cost in async_success_handler

Two places in the logging pipeline were overwriting response_cost that
pass-through handlers (Gemini/Vertex) had already calculated:

1. _process_hidden_params_and_response_cost fell through to
   _response_cost_calculator which returns None for pass-through calls
2. async_success_handler pass-through branch unconditionally set
   response_cost = None (introduced in PR #19887)

Now both places check if response_cost is already set before overwriting.

* test: add regression test for pass-through endpoint response_cost preservation
2026-02-21 15:09:45 -08:00
Ishaan Jaff
3278fee714
fix(lint): fix ruff/flake8 violations - unused imports, PLR0915, print statements (#21846) 2026-02-21 15:07:47 -08:00
ryan-crabbe
77c3b6ab92
Merge pull request #21133 from BerriAI/litellm_perf_convert_model_response_frozensets
perf: cache model_fields.keys() as frozensets
2026-02-21 15:05:26 -08:00
Ishaan Jaff
d427ff6705
fix(logging): zero out response_cost for cache hits in _process_hidden_params_and_response_cost (#21841)
When a cached response had `response_cost` pre-calculated in its `_hidden_params`
(from the original response), `_process_hidden_params_and_response_cost` was using
it directly without checking `cache_hit`, so cached responses were reported with
non-zero cost instead of 0.

Fixes test_cost_tracking_with_caching.
2026-02-21 14:57:03 -08:00
Ryan Crabbe
bb24ebebd9 Merge origin/main into litellm_perf_convert_model_response_frozensets
Resolve conflict: keep main's provider_specific_fields passthrough
preservation while using frozenset set-difference optimization.
2026-02-21 14:52:43 -08:00