litellm/scripts
harish-berri a67b7a7e87
Refactor Bedrock response stream shape handling (#27257)
* Refactor Bedrock response stream shape handling

- Introduced a module-level constant `BEDROCK_RESPONSE_STREAM_SHAPE` to cache the response stream shape, eliminating the need for per-instance caching in `BedrockEventStreamDecoderBase`.
- Updated relevant methods to utilize the new constant, improving performance by avoiding redundant loading of the shape.
- Added tests to ensure the shape is loaded correctly at import time and is consistent across different modules.
- Added a new mock server script for testing Bedrock pass-through functionality.

* Refactor response parsing for Bedrock and SageMaker

- Improved code readability by formatting the parsing method calls in `AWSEventStreamDecoder` for both Bedrock and SageMaker response stream shapes.
- Added blank lines for better separation of code blocks in `invoke_handler.py` and `common_utils.py` to enhance maintainability.

* Enhance error handling for Bedrock and SageMaker response stream shape loading

- Wrapped the loading logic in `_load_bedrock_response_stream_shape` and `_load_sagemaker_response_stream_shape` with try-except blocks to gracefully handle exceptions.
- Added logging to warn when the response stream shape cannot be pre-loaded, ensuring the module imports cleanly.
- Updated tests to verify that loading failures return `None` instead of propagating exceptions.

* Implement error handling for missing response stream shapes in Bedrock and SageMaker

- Added checks in `_parse_message_from_event` methods to raise appropriate errors when `BEDROCK_RESPONSE_STREAM_SHAPE` or `SAGEMAKER_RESPONSE_STREAM_SHAPE` is None, ensuring clearer error reporting.
- Updated logging messages to reflect the unavailability of event-stream decoding for both Bedrock and SageMaker.
- Enhanced unit tests to verify that the correct exceptions are raised when the response stream shapes are not loaded.
2026-05-06 17:39:38 -07:00
..
adaptive_router_demo feat: commit new adaptive routing 2026-04-18 21:29:39 -07:00
health_check style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
benchmark_mock.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
benchmark_proxy_vs_provider.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
create_litellm_branch.ps1 feat: add script to create branches with litellm_ prefix (#17606) 2025-12-06 10:41:39 -08:00
create_litellm_branch.sh enhance: create_litellm_branch tool to be more robust (#17874) 2025-12-12 05:35:50 -08:00
create_team_key_and_submit_guardrail.sh feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
eval_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
install.sh build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
mock_bedrock_passthrough_target.py Refactor Bedrock response stream shape handling (#27257) 2026-05-06 17:39:38 -07:00
mock_grayswan_timeout_server.py implement failopen option default to True on grayswan guardrail (#18266) 2026-01-06 15:17:05 +05:30
test_agent_mcp_endpoints.sh Agents - assign tools (#22064) 2026-02-25 11:44:30 -08:00
test_guardrails_register_endpoints.sh feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
test_tool_allowlist_script.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
tpm_headline_test.sh fix: atomic TPM rate limit (#27001) 2026-05-05 16:58:07 -07:00
verify_adaptive_router.py feat: add adaptive routing to litellm 2026-04-18 16:35:17 -07:00