c23b19f09c
* feat(openai): apply regional-processing cost uplift for EU/US data residency OpenAI charges a 10% uplift on the latest GPT models when requests are served from a regionalized hostname (eu./us.api.openai.com). Infer the region from `api_base`, expose it on `kwargs["litellm_params"]["data_residency"]`, and multiply the computed cost by a per-model `regional_processing_uplift_multiplier_<region>` field. https://claude.ai/code/session_012ebH44s7ohYxjoix5CXzTW * test: allow regional_processing_uplift_multiplier_{eu,us} in model_prices schema * fix(cost): tighten data_residency inference and restore model_cost in tests - Only infer OpenAI data_residency when custom_llm_provider == "openai"; drop the implicit None fallback so non-OpenAI callers can't accidentally pick up a regional tag from a stray OpenAI hostname. - _local_model_cost_map fixture now snapshots and restores litellm.model_cost and LITELLM_LOCAL_MODEL_COST_MAP so tests don't leak state across the session. * refactor(openai): move data_residency helper under llms/openai * fix: thread data_residency through realtime stream cost calculation Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(cost): thread data_residency through batch_cost_calculator Apply the OpenAI regional-processing uplift multiplier to retrieve_batch cost paths so Batch API requests served via eu./us.api.openai.com are priced at the same uplifted token rates as completions/transcriptions. * refactor(openai): encapsulate provider check inside infer_openai_data_residency Move the custom_llm_provider == "openai" guard from get_litellm_params into the helper itself so the core utility no longer carries provider-specific dispatch logic. Callers pass through the provider unconditionally; the helper returns None for any non-OpenAI provider. * fix(responses): thread data_residency through Responses logging params The Responses API paths build their logging litellm_params dict after provider resolution but did not include data_residency, so cost calc saw None even when the effective api_base was a regional OpenAI host. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai>
35 lines
1.1 KiB
Python
35 lines
1.1 KiB
Python
"""Tests for the OpenAI data-residency inference helper."""
|
|
|
|
import pytest
|
|
|
|
from litellm.llms.openai.data_residency import infer_openai_data_residency
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"api_base, expected",
|
|
[
|
|
("https://eu.api.openai.com/v1", "eu"),
|
|
("https://eu.api.openai.com", "eu"),
|
|
("https://us.api.openai.com/v1", "us"),
|
|
("https://us.api.openai.com", "us"),
|
|
("https://EU.api.openai.com/v1", "eu"),
|
|
("https://api.openai.com/v1", None),
|
|
("https://api.openai.com", None),
|
|
("https://example.com/v1", None),
|
|
("https://my-azure-endpoint.openai.azure.com/openai/deployments/foo", None),
|
|
("", None),
|
|
(None, None),
|
|
("not a url", None),
|
|
],
|
|
)
|
|
def test_infer_openai_data_residency(api_base, expected):
|
|
assert infer_openai_data_residency("openai", api_base) == expected
|
|
|
|
|
|
@pytest.mark.parametrize("custom_llm_provider", [None, "anthropic", "azure", "bedrock"])
|
|
def test_infer_openai_data_residency_non_openai_provider(custom_llm_provider):
|
|
assert (
|
|
infer_openai_data_residency(custom_llm_provider, "https://eu.api.openai.com/v1")
|
|
is None
|
|
)
|