ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse (#19545)

Browse Source 
* fix: add AnthropicMessagesRequestOptionalParams

* add _update_headers_with_anthropic_beta

* fix output format tests

* test_structured_output_e2e

* TestAnthropicAPIStructuredOutput

* test_structured_output_e2e

* fix BASE

* TestAzureAnthropicStructuredOutput

* fix: Bedrock Converse

* add nthropic Messages Pass-Through Architecture

* fix: bedrock invoke output_format

* fix: transform_anthropic_messages_request for vertex anthropic

* TestBedrockInvokeStructuredOutput

* docs anthropic vertex

* docs fix

* docs fix
This commit is contained in:
Ishaan Jaff 2026-01-21 20:09:18 -08:00 committed by GitHub
parent 3794f86af0
commit ab606c9a73
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
17 changed files with 726 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
docs/my-website/docs/anthropic_unified.md → docs/my-website/docs/anthropic_unified/index.md
View File

237
docs/my-website/docs/anthropic_unified/structured_output.md Normal file
View File

@ -0,0 +1,237 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Structured Output /v1/messages
Use LiteLLM to call Anthropic's structured output feature via the `/v1/messages` endpoint.
## Supported Providers
| Provider | Supported | Notes |
|----------|-----------|-------|
| Anthropic | ✅ | Native support |
| Azure AI (Anthropic models) | ✅ | Claude models on Azure AI |
| Bedrock (Converse Anthropic models) | ✅ | Claude models via Bedrock Converse API |
## Usage
### LiteLLM Proxy Server
<Tabs>
<TabItem value="anthropic" label="Anthropic">
1. Setup config.yaml
```yaml
model_list:
- model_name: claude-sonnet
litellm_params:
model: anthropic/claude-sonnet-4-5-20250514
api_key: os.environ/ANTHROPIC_API_KEY
```
2. Start proxy
```bash
litellm --config /path/to/config.yaml
```
3. Test it!
```bash
curl http://localhost:4000/v1/messages \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-sonnet",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Extract the key information from this email: John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm."
}
],
"output_format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"plan_interest": {"type": "string"},
"demo_requested": {"type": "boolean"}
},
"required": ["name", "email", "plan_interest", "demo_requested"],
"additionalProperties": false
}
}
}'
```
</TabItem>
<TabItem value="azure_ai" label="Azure AI (Anthropic)">
1. Setup config.yaml
```yaml
model_list:
- model_name: azure-claude-sonnet
litellm_params:
model: azure_ai/claude-sonnet-4-5-20250514
api_key: os.environ/AZURE_AI_API_KEY
api_base: https://your-endpoint.inference.ai.azure.com
```
2. Start proxy
```bash
litellm --config /path/to/config.yaml
```
3. Test it!
```bash
curl http://localhost:4000/v1/messages \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "azure-claude-sonnet",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Extract the key information from this email: John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm."
}
],
"output_format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"plan_interest": {"type": "string"},
"demo_requested": {"type": "boolean"}
},
"required": ["name", "email", "plan_interest", "demo_requested"],
"additionalProperties": false
}
}
}'
```
</TabItem>
<TabItem value="bedrock" label="Bedrock (Converse)">
1. Setup config.yaml
```yaml
model_list:
- model_name: bedrock-claude-sonnet
litellm_params:
model: bedrock/anthropic.claude-sonnet-4-5-20250514-v1:0
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-west-2
```
2. Start proxy
```bash
litellm --config /path/to/config.yaml
```
3. Test it!
```bash
curl http://localhost:4000/v1/messages \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "bedrock-claude-sonnet",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": "Extract the key information from this email: John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm."
}
],
"output_format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"plan_interest": {"type": "string"},
"demo_requested": {"type": "boolean"}
},
"required": ["name", "email", "plan_interest", "demo_requested"],
"additionalProperties": false
}
}
}'
```
</TabItem>
</Tabs>
## Example Response
```json
{
"id": "msg_01XFDUDYJgAACzvnptvVoYEL",
"type": "message",
"role": "assistant",
"content": [
{
"type": "text",
"text": "{\"name\":\"John Smith\",\"email\":\"john@example.com\",\"plan_interest\":\"Enterprise\",\"demo_requested\":true}"
}
],
"model": "claude-sonnet-4-5-20250514",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 75,
"output_tokens": 28
}
}
```
## Request Format
### output_format
The `output_format` parameter specifies the structured output format.
```json
{
"output_format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"field_name": {"type": "string"},
"another_field": {"type": "integer"}
},
"required": ["field_name", "another_field"],
"additionalProperties": false
}
}
}
```
#### Fields
- **type** (string): Must be `"json_schema"`
- **schema** (object): A JSON Schema object defining the expected output structure
- **type** (string): The root type, typically `"object"`
- **properties** (object): Defines the fields and their types
- **required** (array): List of required field names
- **additionalProperties** (boolean): Set to `false` to enforce strict schema adherence

9
docs/my-website/sidebars.js
View File

@ -517,7 +517,14 @@ const sidebars = {
"mcp_troubleshoot",
]
},
"anthropic_unified",
{
type: "category",
label: "/v1/messages",
items: [
"anthropic_unified/index",
"anthropic_unified/structured_output",
]
},
"anthropic_count_tokens",
"moderation",
"ocr",

8
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py
View File

@ -45,6 +45,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools: Optional[List[Dict]] = None,
top_k: Optional[int] = None,
top_p: Optional[float] = None,
output_format: Optional[Dict] = None,
extra_kwargs: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Prepare kwargs for litellm.completion/acompletion"""
@ -76,6 +77,8 @@ class LiteLLMMessagesToCompletionTransformationHandler:
request_data["top_k"] = top_k
if top_p is not None:
request_data["top_p"] = top_p
if output_format:
request_data["output_format"] = output_format
openai_request = ANTHROPIC_ADAPTER.translate_completion_input_params(
request_data
@ -130,6 +133,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools: Optional[List[Dict]] = None,
top_k: Optional[int] = None,
top_p: Optional[float] = None,
output_format: Optional[Dict] = None,
**kwargs,
) -> Union[AnthropicMessagesResponse, AsyncIterator]:
"""Handle non-Anthropic models asynchronously using the adapter"""
@ -148,6 +152,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools=tools,
top_k=top_k,
top_p=top_p,
output_format=output_format,
extra_kwargs=kwargs,
)
)
@ -189,6 +194,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools: Optional[List[Dict]] = None,
top_k: Optional[int] = None,
top_p: Optional[float] = None,
output_format: Optional[Dict] = None,
_is_async: bool = False,
**kwargs,
) -> Union[
@ -212,6 +218,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools=tools,
top_k=top_k,
top_p=top_p,
output_format=output_format,
**kwargs,
)
@ -230,6 +237,7 @@ class LiteLLMMessagesToCompletionTransformationHandler:
tools=tools,
top_k=top_k,
top_p=top_p,
output_format=output_format,
extra_kwargs=kwargs,
)
)

48
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py
View File

@ -172,7 +172,7 @@ class LiteLLMAnthropicMessagesAdapter:
"""
Which anthropic params, we need to translate to the openai format.
"""
return ["messages", "metadata", "system", "tool_choice", "tools", "thinking"]
return ["messages", "metadata", "system", "tool_choice", "tools", "thinking", "output_format"]
def translate_anthropic_messages_to_openai( # noqa: PLR0915
self,
@ -554,6 +554,42 @@ class LiteLLMAnthropicMessagesAdapter:
return new_tools
def translate_anthropic_output_format_to_openai(
self, output_format: Any
) -> Optional[Dict[str, Any]]:
"""
Translate Anthropic's output_format to OpenAI's response_format.
Anthropic output_format: {"type": "json_schema", "schema": {...}}
OpenAI response_format: {"type": "json_schema", "json_schema": {"name": "...", "schema": {...}}}
Args:
output_format: Anthropic output_format dict with 'type' and 'schema'
Returns:
OpenAI-compatible response_format dict, or None if invalid
"""
if not isinstance(output_format, dict):
return None
output_type = output_format.get("type")
if output_type != "json_schema":
return None
schema = output_format.get("schema")
if not schema:
return None
# Convert to OpenAI response_format structure
return {
"type": "json_schema",
"json_schema": {
"name": "structured_output",
"schema": schema,
"strict": True,
},
}
def translate_anthropic_to_openai(
self, anthropic_message_request: AnthropicMessagesRequest
) -> ChatCompletionRequest:
@ -636,6 +672,16 @@ class LiteLLMAnthropicMessagesAdapter:
if reasoning_effort:
new_kwargs["reasoning_effort"] = reasoning_effort
## CONVERT OUTPUT_FORMAT to RESPONSE_FORMAT
if "output_format" in anthropic_message_request:
output_format = anthropic_message_request["output_format"]
if output_format:
response_format = self.translate_anthropic_output_format_to_openai(
output_format=output_format
)
if response_format:
new_kwargs["response_format"] = response_format
translatable_params = self.translatable_anthropic_params()
for k, v in anthropic_message_request.items():
if k not in translatable_params: # pass remaining params as is

51
litellm/llms/anthropic/experimental_pass_through/architecture.md Normal file
View File

@ -0,0 +1,51 @@
# Anthropic Messages Pass-Through Architecture
## Request Flow
```mermaid
flowchart TD
A[litellm.anthropic.messages.acreate] --> B{Provider?}
B -->|anthropic| C[AnthropicMessagesConfig]
B -->|azure_ai| D[AzureAnthropicMessagesConfig]
B -->|bedrock invoke| E[BedrockAnthropicMessagesConfig]
B -->|vertex_ai| F[VertexAnthropicMessagesConfig]
B -->|Other providers| G[LiteLLMAnthropicMessagesAdapter]
C --> H[Direct Anthropic API]
D --> I[Azure AI Foundry API]
E --> J[Bedrock Invoke API]
F --> K[Vertex AI API]
G --> L[translate_anthropic_to_openai]
L --> M[litellm.completion]
M --> N[Provider API]
N --> O[translate_openai_response_to_anthropic]
O --> P[Anthropic Response Format]
H --> P
I --> P
J --> P
K --> P
```
## Adapter Flow (Non-Native Providers)
```mermaid
sequenceDiagram
participant User
participant Handler as anthropic_messages_handler
participant Adapter as LiteLLMAnthropicMessagesAdapter
participant LiteLLM as litellm.completion
participant Provider as Provider API
User->>Handler: Anthropic Messages Request
Handler->>Adapter: translate_anthropic_to_openai()
Note over Adapter: messages, tools, thinking,<br/>output_format → response_format
Adapter->>LiteLLM: OpenAI Format Request
LiteLLM->>Provider: Provider-specific Request
Provider->>LiteLLM: Provider Response
LiteLLM->>Adapter: OpenAI Format Response
Adapter->>Handler: translate_openai_response_to_anthropic()
Handler->>User: Anthropic Messages Response
```

22
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
View File

@ -42,6 +42,7 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
"tool_choice",
"thinking",
"context_management",
"output_format",
# TODO: Add Anthropic `metadata` support
# "metadata",
]
@ -169,27 +170,32 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
) -> dict:
"""
Auto-inject anthropic-beta headers based on features used.
Handles:
- context_management: adds 'context-management-2025-06-27'
- tool_search: adds provider-specific tool search header
- output_format: adds 'structured-outputs-2025-11-13'
Args:
headers: Request headers dict
optional_params: Optional parameters including tools, context_management
optional_params: Optional parameters including tools, context_management, output_format
custom_llm_provider: Provider name for looking up correct tool search header
"""
beta_values: set = set()
# Get existing beta headers if any
existing_beta = headers.get("anthropic-beta")
if existing_beta:
beta_values.update(b.strip() for b in existing_beta.split(","))
# Check for context management
if optional_params.get("context_management") is not None:
beta_values.add(ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value)
# Check for structured outputs
if optional_params.get("output_format") is not None:
beta_values.add(ANTHROPIC_BETA_HEADER_VALUES.STRUCTURED_OUTPUT_2025_09_25.value)
# Check for tool search tools
tools = optional_params.get("tools")
if tools:
@ -198,8 +204,8 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
# Use provider-specific tool search header
tool_search_header = get_tool_search_beta_header(custom_llm_provider)
beta_values.add(tool_search_header)
if beta_values:
headers["anthropic-beta"] = ",".join(sorted(beta_values))
return headers

6
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
View File

@ -271,8 +271,12 @@ class AmazonAnthropicClaudeMessagesConfig(
# 4. Remove `ttl` field from cache_control in messages (Bedrock doesn't support it)
self._remove_ttl_from_cache_control(anthropic_messages_request)
# 5. `output_format` is not supported on Bedrock invoke
if "output_format" in anthropic_messages_request:
anthropic_messages_request.pop("output_format", None)
# 5. AUTO-INJECT beta headers based on features used
# 6. AUTO-INJECT beta headers based on features used
anthropic_model_info = AnthropicModelInfo()
tools = anthropic_messages_optional_request_params.get("tools")
messages_typed = cast(List[AllMessageValues], messages)

5
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py
View File

@ -117,4 +117,9 @@ class VertexAIPartnerModelsAnthropicMessagesConfig(AnthropicMessagesConfig, Vert
anthropic_messages_request.pop(
"model", None
) # do not pass model in request body to vertex ai
anthropic_messages_request.pop(
"output_format", None
) # do not pass output_format in request body to vertex ai - vertex ai does not support output_format as yet
return anthropic_messages_request

1
litellm/types/llms/anthropic.py
View File

@ -359,6 +359,7 @@ class AnthropicMessagesRequestOptionalParams(TypedDict, total=False):
mcp_servers: Optional[List[AnthropicMcpServerTool]]
context_management: Optional[Dict[str, Any]]
container: Optional[Dict[str, Any]] # Container config with skills for code execution
output_format: Optional[AnthropicOutputSchema] # Structured outputs support
class AnthropicMessagesRequest(AnthropicMessagesRequestOptionalParams, total=False):

74
test_anthropic_messages_structured_outputs_minimal.py Normal file
View File

@ -0,0 +1,74 @@
"""
Tests for structured outputs support in Anthropic /v1/messages endpoint.
"""
import pytest
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
)
def test_output_format_supported_and_transforms_correctly():
"""Test that output_format is supported and properly transformed with beta header."""
config = AnthropicMessagesConfig()
# 1. Verify it's in supported parameters
supported_params = config.get_supported_anthropic_messages_params("claude-sonnet-4-5")
assert "output_format" in supported_params
# 2. Verify transformation preserves output_format and adds beta header
output_format = {
"type": "json_schema",
"schema": {"type": "object", "properties": {"result": {"type": "string"}}}
}
optional_params = {"max_tokens": 1024, "output_format": output_format}
headers = {}
# Transform request
result = config.transform_anthropic_messages_request(
model="claude-sonnet-4-5",
messages=[{"role": "user", "content": "test"}],
anthropic_messages_optional_request_params=optional_params.copy(),
litellm_params={},
headers=headers
)
# Update headers
headers = config._update_headers_with_anthropic_beta(headers, optional_params)
# Verify output_format preserved in request body
assert "output_format" in result
assert result["output_format"]["type"] == "json_schema"
# Verify beta header added
assert "anthropic-beta" in headers
assert "structured-outputs-2025-11-13" in headers["anthropic-beta"]
def test_output_format_works_with_bedrock_and_azure():
"""Test that output_format works with Bedrock and Azure Foundry models."""
config = AnthropicMessagesConfig()
output_format = {"type": "json_schema", "schema": {"type": "object", "properties": {}}}
optional_params = {"max_tokens": 1024, "output_format": output_format}
messages = [{"role": "user", "content": "test"}]
# Test Bedrock
bedrock_result = config.transform_anthropic_messages_request(
model="bedrock/anthropic.claude-sonnet-4-5-v2:0",
messages=messages,
anthropic_messages_optional_request_params=optional_params.copy(),
litellm_params={},
headers={}
)
assert "output_format" in bedrock_result
# Test Azure Foundry
azure_result = config.transform_anthropic_messages_request(
model="azure_ai/claude-sonnet-4-5",
messages=messages,
anthropic_messages_optional_request_params=optional_params.copy(),
litellm_params={},
headers={}
)
assert "output_format" in azure_result

12
tests/pass_through_unit_tests/messages_api_structured_output/__init__.py Normal file
View File

@ -0,0 +1,12 @@
"""
Anthropic Messages API Structured Outputs Test Suite
E2E tests for structured outputs functionality across different providers:
- Direct Anthropic API
- Azure AI Foundry Anthropic models
- AWS Bedrock Invoke API
- AWS Bedrock Converse API
All tests validate that the output_format parameter works correctly
and returns valid JSON instead of Markdown text.
"""

138
tests/pass_through_unit_tests/messages_api_structured_output/base_anthropic_messages_structured_output_test.py Normal file
View File

@ -0,0 +1,138 @@
"""
Base test class for Anthropic Messages API structured outputs E2E tests.
Tests that structured outputs work correctly via litellm.anthropic.messages interface
by making actual API calls and validating JSON response format.
"""
import json
import os
import sys
from abc import ABC, abstractmethod
from typing import Any, Dict, List, Optional
sys.path.insert(0, os.path.abspath("../../.."))
import pytest
import litellm
class BaseAnthropicMessagesStructuredOutputTest(ABC):
"""
Base test class for structured outputs E2E tests across different providers.
Subclasses must implement:
- get_model(): Returns the model string to use for tests
Subclasses may optionally implement:
- get_api_base(): Returns the API base URL (for Azure, etc.)
- get_api_key(): Returns the API key (for Azure, etc.)
"""
@abstractmethod
def get_model(self) -> str:
"""
Returns the model string to use for tests.
"""
pass
def get_api_base(self) -> Optional[str]:
"""
Returns the API base URL. Override for providers like Azure.
"""
return None
def get_api_key(self) -> Optional[str]:
"""
Returns the API key. Override for providers like Azure.
"""
return None
def get_output_format_schema(self) -> Dict[str, Any]:
"""
Returns a simple JSON schema for testing structured outputs.
"""
return {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral"]
}
},
"required": ["sentiment"],
"additionalProperties": False
}
}
def get_test_messages(self) -> List[Dict[str, Any]]:
"""
Returns test messages for structured output testing.
"""
return [
{
"role": "user",
"content": "What is the sentiment of this text: 'This product is amazing!' Return only the sentiment."
}
]
@pytest.mark.asyncio
async def test_structured_output_e2e(self):
"""
E2E test: Make actual API call with structured output and validate JSON response.
"""
litellm._turn_on_debug()
messages = self.get_test_messages()
output_format = self.get_output_format_schema()
# Build kwargs with optional api_base and api_key
kwargs: Dict[str, Any] = {
"model": self.get_model(),
"messages": messages,
"max_tokens": 100,
"output_format": output_format,
}
api_base = self.get_api_base()
if api_base:
kwargs["api_base"] = api_base
api_key = self.get_api_key()
if api_key:
kwargs["api_key"] = api_key
response = await litellm.anthropic.messages.acreate(**kwargs)
print(f"Response: {response}")
# Validate response structure - handle both dict and object responses
if isinstance(response, dict):
assert "content" in response
content_list = response["content"]
else:
assert hasattr(response, "content")
content_list = response.content
assert len(content_list) > 0
content = content_list[0]
# Handle both dict and object content blocks
if isinstance(content, dict):
assert "text" in content
response_text = content["text"]
else:
assert hasattr(content, "text")
response_text = content.text
print(f"Response text: {response_text}")
# The response should be valid JSON
parsed_json = json.loads(response_text)
print(f"Parsed JSON: {parsed_json}")
# Validate the JSON structure
assert "sentiment" in parsed_json
assert parsed_json["sentiment"] in ["positive", "negative", "neutral"]

29
tests/pass_through_unit_tests/messages_api_structured_output/test_anthropic_api_structured_output.py Normal file
View File

@ -0,0 +1,29 @@
"""
E2E Test suite for Anthropic API structured outputs via litellm.anthropic.messages.
Tests that structured outputs work correctly with direct Anthropic API calls
by making actual API calls and validating JSON response format.
Requires ANTHROPIC_API_KEY environment variable.
"""
import os
import sys
sys.path.insert(0, os.path.abspath("../../../.."))
from .base_anthropic_messages_structured_output_test import (
BaseAnthropicMessagesStructuredOutputTest,
)
class TestAnthropicAPIStructuredOutput(BaseAnthropicMessagesStructuredOutputTest):
"""
E2E tests for structured outputs with direct Anthropic API.
Uses Claude Sonnet 4.5 which supports structured outputs with the
'anthropic-beta: structured-outputs-2025-11-13' header.
"""
def get_model(self) -> str:
return "claude-sonnet-4-5-20250929"

36
tests/pass_through_unit_tests/messages_api_structured_output/test_azure_anthropic_structured_output.py Normal file
View File

@ -0,0 +1,36 @@
"""
E2E Test suite for Azure Anthropic structured outputs via litellm.anthropic.messages.
Tests that structured outputs work correctly with Azure AI Foundry Anthropic models
by making actual API calls and validating JSON response format.
Requires Azure AI credentials and model deployment.
"""
import os
import sys
from typing import Optional
sys.path.insert(0, os.path.abspath("../../../.."))
from .base_anthropic_messages_structured_output_test import (
BaseAnthropicMessagesStructuredOutputTest,
)
class TestAzureAnthropicStructuredOutput(BaseAnthropicMessagesStructuredOutputTest):
"""
E2E tests for structured outputs with Azure AI Foundry Anthropic models.
Uses the azure_ai/ prefix which routes through Azure AI Foundry
while maintaining the Anthropic Messages API format.
"""
def get_model(self) -> str:
return "azure_ai/claude-opus-4-5"
def get_api_base(self) -> Optional[str]:
return "https://krish-mh44t553-eastus2.services.ai.azure.com/"
def get_api_key(self) -> Optional[str]:
return os.environ.get("AZURE_ANTHROPIC_API_KEY")

29
tests/pass_through_unit_tests/messages_api_structured_output/test_bedrock_converse_structured_output.py Normal file
View File

@ -0,0 +1,29 @@
"""
E2E Test suite for Bedrock Converse API structured outputs via litellm.anthropic.messages.
Tests that structured outputs work correctly with Bedrock Converse API
by making actual API calls and validating JSON response format.
Requires AWS credentials and Bedrock model access.
"""
import os
import sys
sys.path.insert(0, os.path.abspath("../../../.."))
from .base_anthropic_messages_structured_output_test import (
BaseAnthropicMessagesStructuredOutputTest,
)
class TestBedrockConverseStructuredOutput(BaseAnthropicMessagesStructuredOutputTest):
"""
E2E tests for structured outputs with Bedrock Converse API.
Uses the bedrock/converse/ prefix which routes through litellm.completion()
and the AmazonConverseConfig transformation.
"""
def get_model(self) -> str:
return "bedrock/converse/us.anthropic.claude-3-5-sonnet-20241022-v2:0"

32
tests/pass_through_unit_tests/messages_api_structured_output/test_bedrock_invoke_structured_output.py Normal file
View File

@ -0,0 +1,32 @@
"""
E2E Test suite for Bedrock Invoke API structured outputs via litellm.anthropic.messages.
Tests that structured outputs work correctly with Bedrock Invoke API (native Anthropic format)
by making actual API calls and validating JSON response format.
Requires AWS credentials and Bedrock model access.
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.abspath("../../../.."))
from .base_anthropic_messages_structured_output_test import (
BaseAnthropicMessagesStructuredOutputTest,
)
@pytest.mark.skip(reason="Skipping Bedrock Invoke structured output tests")
class TestBedrockInvokeStructuredOutput(BaseAnthropicMessagesStructuredOutputTest):
"""
E2E tests for structured outputs with Bedrock Invoke API.
Uses the bedrock/invoke/ prefix which routes through the native
Anthropic Messages API format on Bedrock.
"""
def get_model(self) -> str:
return "bedrock/invoke/us.anthropic.claude-3-5-sonnet-20241022-v2:0"