litellm/tests/logging_callback_tests
Krish Dholakia c4af2eb5e2
MCP - usage tracking (#12397)
* fix(common_daily_activity.py): initial commit with working mock BE endpoint for mcp usage

* feat(ui/): show mcp server activity on UI

allows admin to know which mcp's are being used

* feat(common_daily_activity.py): return activity by key

* feat(ui/): show top api keys for a given model / mcp server

allow user to know which key is driving spend

* fix(common_daily_activity.py): use known mcp server names

* feat(server.py): log the namespaced tool name (includes server prefix)

allow accurate cost tracking

* feat(db_spend_update_writer.py): log by mcp_namespaced_tool_name

store aggregate daily activity by mcp_namespaced_tool_name

Enables cost / usage tracking by mcp tool name

* fix(server.py): add key/user metadata to mcp calls

* refactor(common_daily_activity.py): update to return mcp activity in API

* fix(common_daily_activity.py): handle empty key

* fix(common_daily_activity.py): track when api key is empty

* test(test_spend_management_endpoints.py): update tests

* fix: fix ui linting error

* fix: fix linting errors

* test: add missing key

* build(schema.prisma): add mcp tool tracking

* fix(migration.sql): add schema migration file

* feat(server.py): add request logging for mcp calls

enables storing the mcp calls

* fix(new_usage.tsx): fix linting errors

* fix: fix code qa errors

* fix(activity_metrics.tsx): fix ui linting errors post-merge

* fix(types/utils.py): fix linting error

* fix(server.py): always have name
2025-07-08 22:08:16 -07:00
..
gcs_pub_sub_body MCP - usage tracking (#12397) 2025-07-08 22:08:16 -07:00
langfuse_expected_request_body fix(utils.py): support non default params for audio transcription (#11212) 2025-05-28 22:24:02 -07:00
base_test.py
conftest.py
create_mock_standard_logging_payload.py
gettysburg.wav (Testing) - Add e2e testing for langfuse logging with tags (#7922) 2025-01-22 09:09:25 -08:00
log.txt test: update test 2025-04-10 14:04:57 -07:00
test_alerting.py [Feat] Performance - Don't create 1 task for every hanging request alert (#11385) 2025-06-03 21:12:54 -07:00
test_amazing_s3_logs.py [Feat] Add failure logging support for s3 logger (#12299) 2025-07-04 11:11:30 -07:00
test_assemble_streaming_responses.py test(test_assemble_streaming_responses.py): update test to use correct type 2025-03-17 18:17:01 -07:00
test_azure_blob_storage.py
test_bedrock_knowledgebase_hook.py [Feat] Add OpenAI Search Vector Store Operation (#12018) 2025-06-24 15:52:43 -07:00
test_built_in_tools_cost_tracking.py test_openai_responses_api_web_search_cost_tracking 2025-05-03 20:22:42 -07:00
test_datadog_llm_obs.py Litellm dev bedrock anthropic 3 7 v2 (#8843) 2025-02-26 16:05:33 -08:00
test_datadog.py Ensure message redaction works for responses API logging (#12291) 2025-07-04 15:11:20 -07:00
test_gcs_pub_sub.py [Feat] Responses API - Add session management support for non-openai models (#10321) 2025-04-25 23:24:24 -07:00
test_generic_api_callback.py fix: pass application/json for GenericAPILogger (#10772) 2025-05-12 14:15:33 -07:00
test_humanloop_unit_tests.py
test_langfuse_e2e_test.py Litellm dev 05 26 2025 p1 (#11165) 2025-05-26 22:00:48 -07:00
test_langfuse_unit_tests.py test: fixes 2025-05-31 12:42:56 -07:00
test_langsmith_unit_test.py (Bug fix) - don't log messages in model_parameters in StandardLoggingPayload (#8932) 2025-03-01 13:39:45 -08:00
test_log_db_redis_services.py
test_logging_redaction_e2e_test.py Ensure message redaction works for responses API logging (#12291) 2025-07-04 15:11:20 -07:00
test_moderations_api_logging.py [Bug Fix] Fix model_group tracked for /v1/messages and /moderations (#11933) 2025-06-20 14:51:50 -07:00
test_opentelemetry_unit_tests.py Fix otel - follow genai semantic conventions + support 'instructions' param for tts (#10608) 2025-05-06 21:57:01 -07:00
test_otel_logging.py Prometheus - fix request increment + add route tracking for streaming requests (#11731) 2025-06-14 16:26:48 -07:00
test_pagerduty_alerting.py [Refactor] Make Pagerduty a free feature (#10857) 2025-05-15 10:12:06 -07:00
test_prometheus_unit_tests.py [Feat] Polish - add better error validation when users configure prometheus metrics and labels to control cardinality (#12182) 2025-07-01 20:17:17 -07:00
test_spend_logs.py [Feat] Responses API - Add session management support for non-openai models (#10321) 2025-04-25 23:24:24 -07:00
test_sqs_logger.py [Feat] Add new AWS SQS Logging Integration (#12176) 2025-06-30 14:02:49 -07:00
test_standard_logging_payload.py [Feat] Responses API - Add session management support for non-openai models (#10321) 2025-04-25 23:24:24 -07:00
test_token_counting.py [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
test_unit_test_litellm_logging.py
test_unit_tests_init_callbacks.py [Feat] Add new AWS SQS Logging Integration (#12176) 2025-06-30 14:02:49 -07:00
test_view_request_resp_logs.py (Feat) - Allow viewing Request/Response Logs stored in GCS Bucket (#8449) 2025-02-10 20:38:55 -08:00