ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Feat] New API Endpoint - Responses API (v1/responses/compact) (#18697)

Browse Source 
* init transform_compact_response_api_request

* init acompact_responses

* init async_compact_response_api_handler in llm http handler

* init transform_compact_response_api_request for openai

* init acompact_responses

* fix acompact_responses

* add OAI Compact API

* docs responses API Compact

* code qa checks

* test_openai_compact_responses_api

* fix mypy linting
This commit is contained in:
Ishaan Jaff 2026-01-06 16:24:04 +05:30 committed by GitHub
parent 2e668d1dfe
commit 76eda472be
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 6289 additions and 1375 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/my-website/docs/proxy/config_settings.md
View File

@ -671,6 +671,7 @@ router_settings:
| LANGSMITH_DEFAULT_RUN_NAME | Default name for Langsmith run
| LANGSMITH_PROJECT | Project name for Langsmith integration
| LANGSMITH_SAMPLING_RATE | Sampling rate for Langsmith logging
| LANGSMITH_TENANT_ID | Tenant ID for Langsmith multi-tenant deployments
| LANGTRACE_API_KEY | API key for Langtrace service
| LASSO_API_BASE | Base URL for Lasso API
| LASSO_API_KEY | API key for Lasso service
@ -776,6 +777,7 @@ router_settings:
| OTEL_EXPORTER_OTLP_HEADERS | Headers for OpenTelemetry requests
| OTEL_SERVICE_NAME | Service name identifier for OpenTelemetry
| OTEL_TRACER_NAME | Tracer name for OpenTelemetry tracing
| OTEL_LOGS_EXPORTER | Exporter type for OpenTelemetry logs (e.g., console)
| PAGERDUTY_API_KEY | API key for PagerDuty Alerting
| PANW_PRISMA_AIRS_API_KEY | API key for PANW Prisma AIRS service
| PANW_PRISMA_AIRS_API_BASE | Base URL for PANW Prisma AIRS service

104
docs/my-website/docs/response_api_compact.md Normal file
View File

@ -0,0 +1,104 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# /responses/compact
Compress conversation history using OpenAI's `/responses/compact` endpoint.
| Feature | Supported |
|---------|-----------|
| Supported LiteLLM Versions | 1.72.0+ |
| Supported Providers | `openai` |
## Usage
### LiteLLM Python SDK
```python showLineNumbers title="Compact Response"
import litellm
response = litellm.compact_responses(
model="openai/gpt-4o",
input=[{"role": "user", "content": "Hello, how are you?"}],
instructions="Be helpful",
previous_response_id="resp_abc123" # optional
)
print(response.id)
print(response.object) # "response.compaction"
print(response.output)
```
### LiteLLM Proxy
<Tabs>
<TabItem value="curl" label="Curl">
```bash showLineNumbers title="Compact Request"
curl http://localhost:4000/v1/responses/compact \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-1234" \
-d '{
"model": "openai/gpt-4o",
"input": [{"role": "user", "content": "Hello"}],
"instructions": "Be helpful"
}'
```
</TabItem>
<TabItem value="openai-sdk" label="OpenAI Python SDK">
```python showLineNumbers title="Compact with OpenAI SDK"
import httpx
response = httpx.post(
"http://localhost:4000/v1/responses/compact",
headers={"Authorization": "Bearer sk-1234"},
json={
"model": "openai/gpt-4o",
"input": [{"role": "user", "content": "Hello"}],
"instructions": "Be helpful"
}
)
print(response.json())
```
</TabItem>
</Tabs>
## Request Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Model to use for compaction |
| `input` | string or array | Yes | Input messages to compact |
| `instructions` | string | No | System instructions |
| `previous_response_id` | string | No | ID of previous response to continue from |
## Response Format
```json
{
"id": "resp_abc123",
"object": "response.compaction",
"created_at": 1734366691,
"output": [
{
"type": "message",
"role": "assistant",
"content": [...]
},
{
"type": "compaction",
"encrypted_content": "..."
}
],
"usage": {
"input_tokens": 100,
"output_tokens": 50,
"total_tokens": 150
}
}
```

9
docs/my-website/sidebars.js
View File

@ -541,7 +541,14 @@ const sidebars = {
},
"realtime",
"rerank",
"response_api",
{
type: "category",
label: "/responses",
items: [
"response_api",
"response_api_compact",
]
},
{
type: "category",
label: "/search",

27
litellm/llms/base_llm/responses/transformation.py
View File

@ -242,3 +242,30 @@ class BaseResponsesAPIConfig(ABC):
#########################################################
########## END CANCEL RESPONSE API TRANSFORMATION #######
#########################################################
#########################################################
########## COMPACT RESPONSE API TRANSFORMATION ##########
#########################################################
@abstractmethod
def transform_compact_response_api_request(
self,
model: str,
input: Union[str, ResponseInputParam],
response_api_optional_request_params: Dict,
api_base: str,
litellm_params: GenericLiteLLMParams,
headers: dict,
) -> Tuple[str, Dict]:
pass
@abstractmethod
def transform_compact_response_api_response(
self,
raw_response: httpx.Response,
logging_obj: LiteLLMLoggingObj,
) -> ResponsesAPIResponse:
pass
#########################################################
########## END COMPACT RESPONSE API TRANSFORMATION ######
#########################################################

170
litellm/llms/custom_httpx/llm_http_handler.py
View File

@ -65,7 +65,6 @@ from litellm.responses.streaming_iterator import (
ResponsesAPIStreamingIterator,
SyncResponsesAPIStreamingIterator,
)
from litellm.types.utils import CallTypes
from litellm.types.containers.main import (
ContainerFileListResponse,
ContainerListResponse,
@ -92,6 +91,7 @@ from litellm.types.rerank import RerankResponse
from litellm.types.responses.main import DeleteResponseResult
from litellm.types.router import GenericLiteLLMParams
from litellm.types.utils import (
CallTypes,
EmbeddingResponse,
FileTypes,
LiteLLMBatch,
@ -3566,6 +3566,174 @@ class BaseLLMHTTPHandler:
logging_obj=logging_obj,
)
def compact_response_api_handler(
self,
model: str,
input: Union[str, "ResponseInputParam"],
responses_api_provider_config: BaseResponsesAPIConfig,
response_api_optional_request_params: Dict,
litellm_params: GenericLiteLLMParams,
logging_obj: LiteLLMLoggingObj,
custom_llm_provider: Optional[str],
extra_headers: Optional[Dict[str, Any]] = None,
extra_body: Optional[Dict[str, Any]] = None,
timeout: Optional[Union[float, httpx.Timeout]] = None,
client: Optional[Union[HTTPHandler, AsyncHTTPHandler]] = None,
_is_async: bool = False,
shared_session: Optional["ClientSession"] = None,
) -> Union[ResponsesAPIResponse, Coroutine[Any, Any, ResponsesAPIResponse]]:
"""
Handler for the compact responses API.
"""
if _is_async:
return self.async_compact_response_api_handler(
model=model,
input=input,
responses_api_provider_config=responses_api_provider_config,
response_api_optional_request_params=response_api_optional_request_params,
litellm_params=litellm_params,
logging_obj=logging_obj,
custom_llm_provider=custom_llm_provider,
extra_headers=extra_headers,
extra_body=extra_body,
timeout=timeout,
client=client,
shared_session=shared_session,
)
if client is None or not isinstance(client, HTTPHandler):
sync_httpx_client = _get_httpx_client(
params={"ssl_verify": litellm_params.get("ssl_verify", None)}
)
else:
sync_httpx_client = client
headers = responses_api_provider_config.validate_environment(
headers=extra_headers or {}, model=model, litellm_params=litellm_params
)
if extra_headers:
headers.update(extra_headers)
api_base = responses_api_provider_config.get_complete_url(
api_base=litellm_params.api_base,
litellm_params=dict(litellm_params),
)
url, data = responses_api_provider_config.transform_compact_response_api_request(
model=model,
input=input,
response_api_optional_request_params=response_api_optional_request_params,
api_base=api_base,
litellm_params=litellm_params,
headers=headers,
)
## LOGGING
logging_obj.pre_call(
input=input,
api_key="",
additional_args={
"complete_input_dict": data,
"api_base": url,
"headers": headers,
},
)
try:
response = sync_httpx_client.post(
url=url, headers=headers, json=data, timeout=timeout
)
except Exception as e:
raise self._handle_error(
e=e,
provider_config=responses_api_provider_config,
)
return responses_api_provider_config.transform_compact_response_api_response(
raw_response=response,
logging_obj=logging_obj,
)
async def async_compact_response_api_handler(
self,
model: str,
input: Union[str, "ResponseInputParam"],
responses_api_provider_config: BaseResponsesAPIConfig,
response_api_optional_request_params: Dict,
litellm_params: GenericLiteLLMParams,
logging_obj: LiteLLMLoggingObj,
custom_llm_provider: Optional[str],
extra_headers: Optional[Dict[str, Any]] = None,
extra_body: Optional[Dict[str, Any]] = None,
timeout: Optional[Union[float, httpx.Timeout]] = None,
client: Optional[Union[HTTPHandler, AsyncHTTPHandler]] = None,
_is_async: bool = False,
shared_session: Optional["ClientSession"] = None,
) -> ResponsesAPIResponse:
"""
Async version of the compact response API handler.
"""
if client is None or not isinstance(client, AsyncHTTPHandler):
verbose_logger.debug(
f"Creating HTTP client for compact_response with shared_session: {id(shared_session) if shared_session else None}"
)
async_httpx_client = get_async_httpx_client(
llm_provider=litellm.LlmProviders(custom_llm_provider),
params={"ssl_verify": litellm_params.get("ssl_verify", None)},
shared_session=shared_session,
)
else:
async_httpx_client = client
headers = responses_api_provider_config.validate_environment(
headers=extra_headers or {}, model=model, litellm_params=litellm_params
)
if extra_headers:
headers.update(extra_headers)
api_base = responses_api_provider_config.get_complete_url(
api_base=litellm_params.api_base,
litellm_params=dict(litellm_params),
)
url, data = responses_api_provider_config.transform_compact_response_api_request(
model=model,
input=input,
response_api_optional_request_params=response_api_optional_request_params,
api_base=api_base,
litellm_params=litellm_params,
headers=headers,
)
## LOGGING
logging_obj.pre_call(
input=input,
api_key="",
additional_args={
"complete_input_dict": data,
"api_base": url,
"headers": headers,
},
)
try:
response = await async_httpx_client.post(
url=url, headers=headers, json=data, timeout=timeout
)
except Exception as e:
raise self._handle_error(
e=e,
provider_config=responses_api_provider_config,
)
return responses_api_provider_config.transform_compact_response_api_response(
raw_response=response,
logging_obj=logging_obj,
)
def list_files(self):
"""
Lists all files

66
litellm/llms/openai/responses/transformation.py
View File

@ -500,3 +500,69 @@ class OpenAIResponsesAPIConfig(BaseResponsesAPIConfig):
response._hidden_params["headers"] = raw_response_headers
return response
#########################################################
########## COMPACT RESPONSE API TRANSFORMATION ##########
#########################################################
def transform_compact_response_api_request(
self,
model: str,
input: Union[str, ResponseInputParam],
response_api_optional_request_params: Dict,
api_base: str,
litellm_params: GenericLiteLLMParams,
headers: dict,
) -> Tuple[str, Dict]:
"""
Transform the compact response API request into a URL and data
OpenAI API expects the following request
- POST /v1/responses/compact
"""
url = f"{api_base}/compact"
input = self._validate_input_param(input)
data = dict(
ResponsesAPIRequestParams(
model=model, input=input, **response_api_optional_request_params
)
)
return url, data
def transform_compact_response_api_response(
self,
raw_response: httpx.Response,
logging_obj: LiteLLMLoggingObj,
) -> ResponsesAPIResponse:
"""
Transform the compact response API response into a ResponsesAPIResponse
"""
try:
logging_obj.post_call(
original_response=raw_response.text,
additional_args={"complete_input_dict": {}},
)
raw_response_json = raw_response.json()
raw_response_json["created_at"] = _safe_convert_created_field(
raw_response_json["created_at"]
)
except Exception:
raise OpenAIError(
message=raw_response.text, status_code=raw_response.status_code
)
raw_response_headers = dict(raw_response.headers)
processed_headers = process_response_headers(raw_response_headers)
try:
response = ResponsesAPIResponse(**raw_response_json)
except Exception:
verbose_logger.debug(
f"Error constructing ResponsesAPIResponse: {raw_response_json}, using model_construct"
)
response = ResponsesAPIResponse.model_construct(**raw_response_json)
response._hidden_params["additional_headers"] = processed_headers
response._hidden_params["headers"] = raw_response_headers
return response

6937
litellm/model_prices_and_context_window_backup.json
View File

File diff suppressed because it is too large Load Diff

2
litellm/proxy/common_request_processing.py
View File

@ -319,6 +319,7 @@ class ProxyBaseLLMRequestProcessing:
"aget_responses",
"adelete_responses",
"acancel_responses",
"acompact_responses",
"acreate_batch",
"aretrieve_batch",
"alist_batches",
@ -457,6 +458,7 @@ class ProxyBaseLLMRequestProcessing:
"aget_responses",
"adelete_responses",
"acancel_responses",
"acompact_responses",
"atext_completion",
"aimage_edit",
"alist_input_items",

6
litellm/proxy/management_endpoints/mcp_management_endpoints.py
View File

@ -32,8 +32,8 @@ from fastapi import (
from fastapi.responses import JSONResponse
import litellm
from litellm._uuid import uuid
from litellm._logging import verbose_logger, verbose_proxy_logger
from litellm._uuid import uuid
from litellm.constants import LITELLM_PROXY_ADMIN_NAME
from litellm.proxy._experimental.mcp_server.utils import (
validate_and_normalize_mcp_server_payload,
@ -67,7 +67,6 @@ if MCP_AVAILABLE:
from litellm.proxy._experimental.mcp_server.ui_session_utils import (
build_effective_auth_contexts,
)
from litellm.proxy.common_utils.http_parsing_utils import _read_request_body
from litellm.proxy._types import (
LiteLLM_MCPServerTable,
LitellmUserRoles,
@ -79,6 +78,7 @@ if MCP_AVAILABLE:
UserMCPManagementMode,
)
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.proxy.common_utils.http_parsing_utils import _read_request_body
from litellm.proxy.management_endpoints.common_utils import _user_has_admin_view
from litellm.proxy.management_helpers.utils import management_endpoint_wrapper
from litellm.types.mcp import MCPCredentials
@ -312,7 +312,7 @@ if MCP_AVAILABLE:
except Exception:
pass
mode = (proxy_general_settings or {}).get("user_mcp_management_mode")
mode = proxy_general_settings.get("user_mcp_management_mode")
if mode == "view_all":
return "view_all"
return "restricted"

82
litellm/proxy/response_api_endpoints/endpoints.py
View File

@ -698,6 +698,88 @@ async def get_response_input_items(
)
@router.post(
"/v1/responses/compact",
dependencies=[Depends(user_api_key_auth)],
tags=["responses"],
)
@router.post(
"/responses/compact",
dependencies=[Depends(user_api_key_auth)],
tags=["responses"],
)
@router.post(
"/openai/v1/responses/compact",
dependencies=[Depends(user_api_key_auth)],
tags=["responses"],
)
async def compact_response(
request: Request,
fastapi_response: Response,
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
):
"""
Compact a response by running a compaction pass over a conversation.
Returns encrypted, opaque items that can be used to reduce context size.
Follows the OpenAI Responses API spec: https://platform.openai.com/docs/api-reference/responses/compact
```bash
curl -X POST http://localhost:4000/v1/responses/compact \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-1234" \
-d '{
"model": "gpt-4o",
"input": [{"role": "user", "content": "Hello"}]
}'
```
"""
from litellm.proxy.proxy_server import (
_read_request_body,
general_settings,
llm_router,
proxy_config,
proxy_logging_obj,
select_data_generator,
user_api_base,
user_max_tokens,
user_model,
user_request_timeout,
user_temperature,
version,
)
data = await _read_request_body(request=request)
processor = ProxyBaseLLMRequestProcessing(data=data)
try:
return await processor.base_process_llm_request(
request=request,
fastapi_response=fastapi_response,
user_api_key_dict=user_api_key_dict,
route_type="acompact_responses",
proxy_logging_obj=proxy_logging_obj,
llm_router=llm_router,
general_settings=general_settings,
proxy_config=proxy_config,
select_data_generator=select_data_generator,
model=None,
user_model=user_model,
user_temperature=user_temperature,
user_request_timeout=user_request_timeout,
user_max_tokens=user_max_tokens,
user_api_base=user_api_base,
version=version,
)
except Exception as e:
raise await processor._handle_llm_api_exception(
e=e,
user_api_key_dict=user_api_key_dict,
proxy_logging_obj=proxy_logging_obj,
version=version,
)
@router.post(
"/v1/responses/{response_id}/cancel",
dependencies=[Depends(user_api_key_auth)],

2
litellm/proxy/route_llm_request.py
View File

@ -25,6 +25,7 @@ ROUTE_ENDPOINT_MAPPING = {
"alist_input_items": "/responses/{response_id}/input_items",
"aimage_edit": "/images/edits",
"acancel_responses": "/responses/{response_id}/cancel",
"acompact_responses": "/responses/compact",
"aocr": "/ocr",
"asearch": "/search",
"avideo_generation": "/videos",
@ -116,6 +117,7 @@ async def route_request(
"aget_responses",
"adelete_responses",
"acancel_responses",
"acompact_responses",
"acreate_response_reply",
"alist_input_items",
"_arealtime", # private function for realtime API

202
litellm/responses/main.py
View File

@ -1361,3 +1361,205 @@ def cancel_responses(
completion_kwargs=local_vars,
extra_kwargs=kwargs,
)
@client
async def acompact_responses(
input: Union[str, ResponseInputParam],
model: str,
instructions: Optional[str] = None,
previous_response_id: Optional[str] = None,
# Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs.
# The extra values given here take precedence over values defined on the client or passed to this method.
extra_headers: Optional[Dict[str, Any]] = None,
extra_query: Optional[Dict[str, Any]] = None,
extra_body: Optional[Dict[str, Any]] = None,
timeout: Optional[Union[float, httpx.Timeout]] = None,
# LiteLLM specific params,
custom_llm_provider: Optional[str] = None,
**kwargs,
) -> ResponsesAPIResponse:
"""
Async version of the POST Compact Responses API
POST /v1/responses/compact endpoint in the responses API
Runs a compaction pass over a conversation, returning encrypted, opaque items.
"""
local_vars = locals()
try:
loop = asyncio.get_event_loop()
kwargs["acompact_responses"] = True
# get custom llm provider so we can use this for mapping exceptions
if custom_llm_provider is None:
_, custom_llm_provider, _, _ = litellm.get_llm_provider(
model=model, api_base=local_vars.get("base_url", None)
)
func = partial(
compact_responses,
input=input,
model=model,
instructions=instructions,
previous_response_id=previous_response_id,
extra_headers=extra_headers,
extra_query=extra_query,
extra_body=extra_body,
timeout=timeout,
custom_llm_provider=custom_llm_provider,
**kwargs,
)
ctx = contextvars.copy_context()
func_with_context = partial(ctx.run, func)
init_response = await loop.run_in_executor(None, func_with_context)
if asyncio.iscoroutine(init_response):
response = await init_response
else:
response = init_response
# Update the responses_api_response_id with the model_id
if isinstance(response, ResponsesAPIResponse):
response = ResponsesAPIRequestUtils._update_responses_api_response_id_with_model_id(
responses_api_response=response,
litellm_metadata=kwargs.get("litellm_metadata", {}),
custom_llm_provider=custom_llm_provider,
)
return response
except Exception as e:
raise litellm.exception_type(
model=model,
custom_llm_provider=custom_llm_provider,
original_exception=e,
completion_kwargs=local_vars,
extra_kwargs=kwargs,
)
@client
def compact_responses(
input: Union[str, ResponseInputParam],
model: str,
instructions: Optional[str] = None,
previous_response_id: Optional[str] = None,
# Use the following arguments if you need to pass additional parameters to the API that aren't available via kwargs.
# The extra values given here take precedence over values defined on the client or passed to this method.
extra_headers: Optional[Dict[str, Any]] = None,
extra_query: Optional[Dict[str, Any]] = None,
extra_body: Optional[Dict[str, Any]] = None,
timeout: Optional[Union[float, httpx.Timeout]] = None,
# LiteLLM specific params,
custom_llm_provider: Optional[str] = None,
**kwargs,
) -> Union[ResponsesAPIResponse, Coroutine[Any, Any, ResponsesAPIResponse]]:
"""
Synchronous version of the POST Compact Responses API
POST /v1/responses/compact endpoint in the responses API
Runs a compaction pass over a conversation, returning encrypted, opaque items.
"""
local_vars = locals()
try:
litellm_logging_obj: LiteLLMLoggingObj = kwargs.get("litellm_logging_obj") # type: ignore
litellm_call_id: Optional[str] = kwargs.get("litellm_call_id", None)
_is_async = kwargs.pop("acompact_responses", False) is True
# get llm provider logic
litellm_params = GenericLiteLLMParams(**kwargs)
(
model,
custom_llm_provider,
dynamic_api_key,
dynamic_api_base,
) = litellm.get_llm_provider(
model=model,
custom_llm_provider=custom_llm_provider,
api_base=litellm_params.api_base,
api_key=litellm_params.api_key,
)
if custom_llm_provider is None:
raise ValueError("custom_llm_provider is required but passed as None")
# get provider config
responses_api_provider_config: Optional[BaseResponsesAPIConfig] = (
ProviderConfigManager.get_provider_responses_api_config(
model=model,
provider=litellm.LlmProviders(custom_llm_provider),
)
)
if responses_api_provider_config is None:
raise ValueError(
f"COMPACT responses is not supported for {custom_llm_provider}"
)
local_vars.update(kwargs)
# Build optional params for compact endpoint
response_api_optional_params: ResponsesAPIOptionalRequestParams = (
ResponsesAPIRequestUtils.get_requested_response_api_optional_param(
local_vars
)
)
# Get optional parameters for the responses API
responses_api_request_params: Dict = (
ResponsesAPIRequestUtils.get_optional_params_responses_api(
model=model,
responses_api_provider_config=responses_api_provider_config,
response_api_optional_params=response_api_optional_params,
allowed_openai_params=None,
)
)
# Pre Call logging
litellm_logging_obj.update_environment_variables(
model=model,
optional_params=dict(responses_api_request_params),
litellm_params={
**responses_api_request_params,
"litellm_call_id": litellm_call_id,
},
custom_llm_provider=custom_llm_provider,
)
# Call the handler with _is_async flag instead of directly calling the async handler
response = base_llm_http_handler.compact_response_api_handler(
model=model,
input=input,
responses_api_provider_config=responses_api_provider_config,
response_api_optional_request_params=responses_api_request_params,
litellm_params=litellm_params,
logging_obj=litellm_logging_obj,
custom_llm_provider=custom_llm_provider,
extra_headers=extra_headers,
extra_body=extra_body,
timeout=timeout or request_timeout,
_is_async=_is_async,
client=kwargs.get("client"),
shared_session=kwargs.get("shared_session"),
)
# Update the responses_api_response_id with the model_id
if isinstance(response, ResponsesAPIResponse):
response = ResponsesAPIRequestUtils._update_responses_api_response_id_with_model_id(
responses_api_response=response,
litellm_metadata=kwargs.get("litellm_metadata", {}),
custom_llm_provider=custom_llm_provider,
)
return response
except Exception as e:
raise litellm.exception_type(
model=model,
custom_llm_provider=custom_llm_provider,
original_exception=e,
completion_kwargs=local_vars,
extra_kwargs=kwargs,
)

5
litellm/router.py
View File

@ -829,6 +829,9 @@ class Router:
self.acancel_responses = self.factory_function(
litellm.acancel_responses, call_type="acancel_responses"
)
self.acompact_responses = self.factory_function(
litellm.acompact_responses, call_type="acompact_responses"
)
self.adelete_responses = self.factory_function(
litellm.adelete_responses, call_type="adelete_responses"
)
@ -3941,6 +3944,7 @@ class Router:
"anthropic_messages",
"aresponses",
"acancel_responses",
"acompact_responses",
"responses",
"aget_responses",
"adelete_responses",
@ -4169,6 +4173,7 @@ class Router:
elif call_type in (
"aget_responses",
"acancel_responses",
"acompact_responses",
"adelete_responses",
"alist_input_items",
):

4
provider_endpoints_support.json
View File

@ -28,7 +28,8 @@
"list_container_files": "Supports GET /containers/{id}/files endpoint",
"retrieve_container_file": "Supports GET /containers/{id}/files/{file_id} endpoint",
"retrieve_container_file_content": "Supports GET /containers/{id}/files/{file_id}/content endpoint",
"delete_container_file": "Supports DELETE /containers/{id}/files/{file_id} endpoint"
"delete_container_file": "Supports DELETE /containers/{id}/files/{file_id} endpoint",
"compact": "Supports /responses/compact endpoint"
}
}
},
@ -1519,6 +1520,7 @@
"retrieve_container_file": true,
"retrieve_container_file_content": true,
"delete_container_file": true,
"compact": true,
"a2a": true,
"interactions": true
}

46
tests/llm_responses_api_testing/test_openai_responses_api.py
View File

@ -1814,3 +1814,49 @@ async def test_extra_body_merges_with_request_data(extra_body_mock_response_data
assert "temperature" in request_body
assert "custom_field" in request_body
assert request_body["custom_field"] == "custom_value"
@pytest.mark.asyncio
@pytest.mark.parametrize("sync_mode", [True, False])
async def test_openai_compact_responses_api(sync_mode):
"""
Test the compact_responses API for OpenAI.
This test verifies that the compact_responses endpoint works correctly
for compressing conversation history.
"""
litellm._turn_on_debug()
litellm.set_verbose = True
input_messages = [
{"role": "user", "content": "Hello, how are you?"},
{"role": "assistant", "content": "I'm doing well, thank you for asking!"},
{"role": "user", "content": "What is the weather like today?"},
]
try:
if sync_mode:
response = litellm.compact_responses(
model="openai/gpt-4o",
input=input_messages,
instructions="Be helpful and concise",
)
else:
response = await litellm.acompact_responses(
model="openai/gpt-4o",
input=input_messages,
instructions="Be helpful and concise",
)
except litellm.InternalServerError:
pytest.skip("Skipping test due to InternalServerError")
except litellm.BadRequestError as e:
# compact_responses may not be available for all models/accounts
pytest.skip(f"Skipping test due to BadRequestError: {e}")
print("compact_responses response=", json.dumps(response, indent=4, default=str))
# Validate response structure
assert response is not None
assert "id" in response, "Response should have an 'id' field"
assert "output" in response, "Response should have an 'output' field"
assert isinstance(response["output"], list), "Output should be a list"