ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(callbacks): forward callback_settings to callback initializers and guard consumers against non-dict values (#30161)

Browse Source 
* fix(datadog): pass callback_specific_params so DatadogCostManagementLogger receives cost_tag_keys (#29590)

* fix(datadog): pass callback_specific_params so DatadogCostManagementLogger receives cost_tag_keys

* test(proxy): regression test that load_config forwards callback_specific_params

* fix(proxy): guard lakera_prompt_injection callback_specific_params against non-dict

Addresses review feedback: forwarding callback_settings as callback_specific_params
(so DatadogCostManagementLogger receives cost_tag_keys) exposed the
lakera_prompt_injection branch, which did lakeraAI_Moderation(**callback_specific_params
["lakera_prompt_injection"]) with no type guard. A config like
`callback_settings: {lakera_prompt_injection: "any-string"}` then hit `**"any-string"`
-> TypeError: argument after ** must be a mapping, not str.

Guard the lakera branch with isinstance(dict), matching the existing presidio and
datadog_cost_management branches (non-dict values fall back to {}). Add a regression
test asserting initialize_callbacks_on_proxy ignores a non-dict value instead of crashing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: inject fake lakera_ai module to avoid importing the real one

CI fix for the lakera regression test: it stubbed litellm.proxy.proxy_server with
a SimpleNamespace and then monkeypatch.setattr'd the real lakera_ai module, which
forces importing it — and lakera_ai does `from litellm.proxy.proxy_server import
LiteLLM_TeamTable`, absent on the stub -> ImportError under proxy-infra tests.

Inject a fake lakera_ai module into sys.modules instead, so the callbacks branch's
`from ...lakera_ai import lakeraAI_Moderation` resolves to the stub without loading
the real module. The guard under test (isinstance(dict) in the lakera branch) is
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(callbacks): guard compression/websearch interceptors against non-dict callback_settings (#30153)

#29590 forwards the full callback_settings dict into initialize_callbacks_on_proxy, which activates the compression_interception and websearch_interception consumers. Their initialize_from_proxy_config read the callback_settings subkey without an isinstance(dict) guard, so a non-dict value such as `compression_interception: true` reached from_config_yaml(...).get(...) and aborted proxy startup with AttributeError. #29590 added that guard for lakera_prompt_injection but not for these two

Mirror the isinstance(dict) guard already used by the lakera, presidio, and datadog branches so a non-dict value is ignored and the callback initializes with defaults. A parametrized test feeds every callback_settings consumer a non-dict value through initialize_callbacks_on_proxy to catch a future consumer that forgets the guard

* fix(callbacks): normalize non-dict callback_specific_params to empty dict

A blank callback_settings: key in YAML loads as None, and
config.get('callback_settings', {}) returns None because dict.get only
falls back to the default when the key is absent. Forwarding that value
verbatim to initialize_callbacks_on_proxy made the first
'<name>' in callback_specific_params membership test raise
TypeError: argument of type 'NoneType' is not iterable, aborting proxy
startup. Same failure for any non-dict root such as callback_settings: true.

Normalize the value at the function boundary so both callsites (and any
future ones) initialize callbacks with their defaults instead of crashing.

---------

Co-authored-by: Hedi Daoud <150018939+hdaoud23@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
yuneng-jiang 2026-06-10 15:22:00 -07:00 committed by GitHub
parent 20e453f698
commit 7899463c6a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
8 changed files with 277 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
litellm/integrations/compression_interception/handler.py
View File

@ -72,8 +72,13 @@ class CompressionInterceptionLogger(CustomLogger):
compression_params: CompressionInterceptionConfig = {}
if "compression_interception_params" in litellm_settings:
compression_params = litellm_settings["compression_interception_params"]
elif "compression_interception" in callback_specific_params:
compression_params = callback_specific_params["compression_interception"]
elif "compression_interception" in callback_specific_params and isinstance(
callback_specific_params["compression_interception"], dict
):
compression_params = cast(
CompressionInterceptionConfig,
callback_specific_params["compression_interception"],
)
return CompressionInterceptionLogger.from_config_yaml(compression_params)
async def async_pre_call_deployment_hook(

9
litellm/integrations/websearch_interception/handler.py
View File

@ -1339,8 +1339,13 @@ class WebSearchInterceptionLogger(CustomLogger):
websearch_params: WebSearchInterceptionConfig = {}
if "websearch_interception_params" in litellm_settings:
websearch_params = litellm_settings["websearch_interception_params"]
elif "websearch_interception" in callback_specific_params:
websearch_params = callback_specific_params["websearch_interception"]
elif "websearch_interception" in callback_specific_params and isinstance(
callback_specific_params["websearch_interception"], dict
):
websearch_params = cast(
WebSearchInterceptionConfig,
callback_specific_params["websearch_interception"],
)
# Use classmethod to initialize from config
return WebSearchInterceptionLogger.from_config_yaml(websearch_params)

11
litellm/proxy/common_utils/callback_utils.py
View File

@ -40,8 +40,10 @@ def initialize_callbacks_on_proxy( # noqa: PLR0915
premium_user: bool,
config_file_path: str,
litellm_settings: dict,
callback_specific_params: dict = {},
callback_specific_params: Optional[dict] = None,
):
if not isinstance(callback_specific_params, dict):
callback_specific_params = {}
from litellm.integrations.custom_logger import CustomLogger
from litellm.litellm_core_utils.logging_callback_manager import (
LoggingCallbackManager,
@ -166,7 +168,12 @@ def initialize_callbacks_on_proxy( # noqa: PLR0915
)
init_params = {}
if "lakera_prompt_injection" in callback_specific_params:
if (
"lakera_prompt_injection" in callback_specific_params
and isinstance(
callback_specific_params["lakera_prompt_injection"], dict
)
):
init_params = callback_specific_params["lakera_prompt_injection"]
lakera_moderations_object = lakeraAI_Moderation(**init_params)
imported_list.append(lakera_moderations_object)

1
litellm/proxy/proxy_server.py
View File

@ -4079,6 +4079,7 @@ class ProxyConfig:
premium_user=premium_user,
config_file_path=config_file_path,
litellm_settings=litellm_settings,
callback_specific_params=callback_settings,
)
elif key == "model_group_settings":

34
tests/test_litellm/integrations/compression_interception/test_compression_interception_handler.py
View File

@ -32,6 +32,40 @@ def test_initialize_from_proxy_config():
assert logger.compression_target == 789
def test_initialize_from_proxy_config_ignores_non_dict_callback_specific_params():
"""Regression (#29590): a non-dict value under
callback_settings.compression_interception must not crash initialization.
Forwarding callback_settings as callback_specific_params activates this
branch; without the isinstance(dict) guard a non-dict value reached
from_config_yaml(...).get(...) and raised AttributeError at proxy startup.
The value is ignored and the logger falls back to defaults.
"""
logger = CompressionInterceptionLogger.initialize_from_proxy_config(
litellm_settings={},
callback_specific_params={"compression_interception": True},
)
assert logger.enabled is True
assert logger.compression_trigger == 200_000
def test_initialize_from_proxy_config_honors_dict_callback_specific_params():
"""A valid dict under callback_settings.compression_interception is applied."""
logger = CompressionInterceptionLogger.initialize_from_proxy_config(
litellm_settings={},
callback_specific_params={
"compression_interception": {
"enabled": False,
"compression_trigger": 12345,
}
},
)
assert logger.enabled is False
assert logger.compression_trigger == 12345
@pytest.mark.asyncio
async def test_pre_call_hook_compresses_messages_and_injects_tool(monkeypatch):
"""Test pre-call hook compresses and stores per-call cache."""

29
tests/test_litellm/integrations/websearch_interception/test_websearch_interception_handler.py
View File

@ -34,6 +34,35 @@ def test_initialize_from_proxy_config():
assert logger.search_tool_name == "my-search"
def test_initialize_from_proxy_config_ignores_non_dict_callback_specific_params():
"""Regression (#29590): a non-dict value under
callback_settings.websearch_interception must not crash initialization.
Forwarding callback_settings as callback_specific_params activates this
branch; without the isinstance(dict) guard a non-dict value reached
from_config_yaml(...).get(...) and raised AttributeError at proxy startup.
The value is ignored and the logger falls back to defaults.
"""
logger = WebSearchInterceptionLogger.initialize_from_proxy_config(
litellm_settings={},
callback_specific_params={"websearch_interception": True},
)
assert logger.search_tool_name is None
def test_initialize_from_proxy_config_honors_dict_callback_specific_params():
"""A valid dict under callback_settings.websearch_interception is applied."""
logger = WebSearchInterceptionLogger.initialize_from_proxy_config(
litellm_settings={},
callback_specific_params={
"websearch_interception": {"search_tool_name": "ws-tool"}
},
)
assert logger.search_tool_name == "ws-tool"
@pytest.mark.asyncio
async def test_async_should_run_agentic_loop():
"""Test that agentic loop is NOT triggered for wrong provider or missing WebSearch tool"""

99
tests/test_litellm/proxy/common_utils/test_callback_utils.py
View File

@ -1,7 +1,9 @@
import copy
import sys
import os
from types import SimpleNamespace
from types import ModuleType, SimpleNamespace
import pytest
sys.path.insert(
0, os.path.abspath("../../..")
@ -309,3 +311,98 @@ def test_encrypt_callback_vars_only_encrypts_credential_fields(monkeypatch):
assert cv["langfuse_host"] == "https://cloud.langfuse.com"
assert cv["langsmith_project"] == "my-proj"
assert cv["langsmith_base_url"] == "https://smith.example"
def test_initialize_callbacks_on_proxy_lakera_ignores_non_dict_callback_settings(
monkeypatch,
):
"""Regression: a non-dict value under callback_settings.lakera_prompt_injection
must not crash initialize_callbacks_on_proxy.
Forwarding callback_settings as callback_specific_params (so callbacks like
DatadogCostManagementLogger receive their init params) exposes the lakera
branch, which previously did lakeraAI_Moderation(**callback_specific_params[
"lakera_prompt_injection"]) with no isinstance(dict) guard. For a config like
{"lakera_prompt_injection": "x"} that is `**"x"` -> TypeError: argument after
** must be a mapping, not str. The branch now guards on isinstance(dict),
matching the presidio / datadog_cost_management branches.
"""
captured = {}
class _DummyLakera:
def __init__(self, **kwargs):
captured["kwargs"] = kwargs
# Inject a fake lakera_ai module so the branch's
# `from ...lakera_ai import lakeraAI_Moderation` resolves to our stub without
# importing the real module (which imports proxy_server symbols not present
# under the stubbed proxy_server below).
fake_lakera = ModuleType("litellm.proxy.guardrails.guardrail_hooks.lakera_ai")
fake_lakera.lakeraAI_Moderation = _DummyLakera
monkeypatch.setitem(
sys.modules,
"litellm.proxy.guardrails.guardrail_hooks.lakera_ai",
fake_lakera,
)
monkeypatch.setitem(
sys.modules,
"litellm.proxy.proxy_server",
SimpleNamespace(prisma_client=None),
)
original_callbacks = (
list(litellm.callbacks) if isinstance(litellm.callbacks, list) else []
)
litellm.callbacks = []
try:
# A non-dict value must be ignored (init_params stays {}), not **-unpacked.
initialize_callbacks_on_proxy(
value=["lakera_prompt_injection"],
premium_user=False,
config_file_path=".",
litellm_settings={},
callback_specific_params={"lakera_prompt_injection": "any-string"},
)
assert captured["kwargs"] == {}
assert any(isinstance(c, _DummyLakera) for c in litellm.callbacks)
finally:
litellm.callbacks = original_callbacks
@pytest.mark.parametrize("bad_root", [None, True])
def test_initialize_callbacks_on_proxy_non_dict_callback_specific_params_root(
monkeypatch, bad_root
):
"""Regression: a blank `callback_settings:` key in YAML loads as None (and
`callback_settings: true` as a bool); load_config forwards that value
verbatim as callback_specific_params. Membership tests like
`"compression_interception" in callback_specific_params` then raise
TypeError and abort proxy startup. A non-dict root must be normalized to {}
so the callback initializes with its defaults.
"""
monkeypatch.setitem(
sys.modules,
"litellm.proxy.proxy_server",
SimpleNamespace(prisma_client=None),
)
from litellm.integrations.compression_interception.handler import (
CompressionInterceptionLogger,
)
original_callbacks = (
list(litellm.callbacks) if isinstance(litellm.callbacks, list) else []
)
litellm.callbacks = []
try:
initialize_callbacks_on_proxy(
value=["compression_interception"],
premium_user=False,
config_file_path=".",
litellm_settings={},
callback_specific_params=bad_root,
)
assert any(
isinstance(c, CompressionInterceptionLogger) for c in litellm.callbacks
)
finally:
litellm.callbacks = original_callbacks

92
tests/test_litellm/proxy/proxy_server/test_proxy_config.py
View File

@ -601,6 +601,98 @@ async def test_ProxyConfig_load_config_missing_file_raises(monkeypatch):
await pc.load_config(router=None, config_file_path="/no/file.yaml")
@pytest.mark.asyncio
async def test_ProxyConfig_load_config_forwards_callback_specific_params(
tmp_path, monkeypatch
):
"""Regression: callback_settings from config must be forwarded to
initialize_callbacks_on_proxy as callback_specific_params.
Callbacks like DatadogCostManagementLogger read their init params (e.g.
cost_tag_keys) from callback_specific_params[<callback_name>]. If the
argument is dropped at the call site, they silently initialize with empty
params and the configured allowlist never takes effect.
"""
f = tmp_path / "c.yaml"
f.write_text(
"model_list: []\n"
"general_settings: {}\n"
"callback_settings:\n"
" datadog_cost_management:\n"
" cost_tag_keys:\n"
" - capability\n"
" - platform\n"
" - ai_product\n"
"litellm_settings:\n"
' callbacks: ["datadog_cost_management"]\n'
)
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", None)
monkeypatch.setattr("litellm.proxy.proxy_server.store_model_in_db", False)
monkeypatch.delenv("LITELLM_CONFIG_BUCKET_NAME", raising=False)
captured = {}
def _fake_initialize_callbacks_on_proxy(**kwargs):
captured.update(kwargs)
monkeypatch.setattr(
"litellm.proxy.proxy_server.initialize_callbacks_on_proxy",
_fake_initialize_callbacks_on_proxy,
)
pc = ProxyConfig()
await pc.load_config(router=None, config_file_path=str(f))
# The callbacks branch must forward the loaded callback_settings.
assert captured.get("callback_specific_params") == {
"datadog_cost_management": {
"cost_tag_keys": ["capability", "platform", "ai_product"]
}
}
@pytest.mark.asyncio
async def test_ProxyConfig_load_config_blank_callback_settings_does_not_crash(
tmp_path, monkeypatch
):
"""Regression: `callback_settings:` with no body loads as None because
dict.get() only falls back to the default when the key is absent. The None
was forwarded verbatim to initialize_callbacks_on_proxy, where the first
`"<name>" in callback_specific_params` membership test raised
TypeError: argument of type 'NoneType' is not iterable, aborting startup.
Startup must succeed and the callback must initialize with its defaults.
"""
f = tmp_path / "c.yaml"
f.write_text(
"model_list: []\n"
"general_settings: {}\n"
"callback_settings:\n"
"litellm_settings:\n"
' callbacks: ["compression_interception"]\n'
)
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", None)
monkeypatch.setattr("litellm.proxy.proxy_server.store_model_in_db", False)
monkeypatch.delenv("LITELLM_CONFIG_BUCKET_NAME", raising=False)
from litellm.integrations.compression_interception.handler import (
CompressionInterceptionLogger,
)
original_callbacks = (
list(litellm.callbacks) if isinstance(litellm.callbacks, list) else []
)
litellm.callbacks = []
try:
pc = ProxyConfig()
await pc.load_config(router=None, config_file_path=str(f))
assert any(
isinstance(c, CompressionInterceptionLogger) for c in litellm.callbacks
)
finally:
litellm.callbacks = original_callbacks
# ---------------------------------------------------------------------------
# ProxyConfig._init_non_llm_configs
# ---------------------------------------------------------------------------