litellm/tests/llm_responses_api_testing
Ishaan Jaff d11832bfad
fix(responses): eliminate per-chunk thread spawning in async streaming path (#21709)
* fix(responses): fix O(n²) CPU overhead in reasoning streaming path

stream_chunk_builder was called on every reasoning chunk, rebuilding the
entire response from all collected chunks each time. Replace with
incremental accumulation of reasoning_content parts, only joining at
reasoning end.

* fix(responses): eliminate per-chunk thread spawning in async streaming path

_process_chunk() called run_async_function() on every SSE chunk, which
when invoked from an async context spawns a thread + event loop per call.

Move the hook call out of _process_chunk into the callers: async __anext__
directly awaits it, sync __next__ uses run_async_function.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: reduce responses streaming CPU for text-only streams

* fix(test): replace deprecated claude-3-7-sonnet-latest in responses API test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result fix test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): replace deprecated claude-3-7-sonnet-latest in tool result empty call_id test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:26:23 -08:00
..
base_responses_api.py fixing test_basic_openai_responses_api 2026-02-16 20:14:06 -08:00
conftest.py [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 2025-08-23 13:13:23 -07:00
test_anthropic_responses_api.py fix(responses): eliminate per-chunk thread spawning in async streaming path (#21709) 2026-02-20 16:26:23 -08:00
test_anthropic_tool_result_empty_call_id.py fix(responses): eliminate per-chunk thread spawning in async streaming path (#21709) 2026-02-20 16:26:23 -08:00
test_anthropic_tool_result_fix.py fix(responses): eliminate per-chunk thread spawning in async streaming path (#21709) 2026-02-20 16:26:23 -08:00
test_azure_responses_api.py def get_advanced_model_for_shell_tool(self) -> Optional[str]: 2026-02-14 12:29:42 -08:00
test_base_responses_api_streaming_iterator.py Fix: Responses API logging error for StopIteration 2026-01-23 18:42:33 +05:30
test_google_ai_studio_responses_api.py Litellm sameer nov 3 stable branch (#16963) 2025-11-22 09:35:05 -08:00
test_manus_files_all_methods.py [Feat] Manus FILES API - Add File upload, get, delete, list (#18904) 2026-01-10 13:27:54 -08:00
test_manus_responses_api.py test_manus_responses_api_with_file_upload 2026-01-10 15:12:00 -08:00
test_openai_responses_api.py [Feat] Adds Shell tool support for the OpenAI Responses API (#21063) 2026-02-12 13:04:29 -08:00
test_responses_hooks.py Align responses API streaming hooks with chat pipeline 2026-01-06 10:54:19 +08:00