litellm/tests/llm_translation
Shivaang d524b79574
fix(types): normalize null top_logprobs to [] in ChatCompletionTokenLogprob (#22245)
Some OpenAI-compatible providers return null for top_logprobs when
logprobs=true but top_logprobs is unset or 0. The OpenAI spec requires
top_logprobs to be Array<TopLogprob> (never null), so this triggers
Pydantic validation errors while parsing responses.

Add a Pydantic v2 field_validator on ChatCompletionTokenLogprob that
normalizes None -> [] before type validation. This preserves the typed
List[TopLogprob] contract for downstream consumers while remaining
narrowly scoped to null only (other invalid types are still rejected).

Fixes #21932
2026-02-28 15:59:29 -08:00
..
fixtures
realtime [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_llm_response_utils fix(types): normalize null top_logprobs to [] in ChatCompletionTokenLogprob (#22245) 2026-02-28 15:59:29 -08:00
test_skills_data Remove Apache 2 license from SKILL.md (#22322) 2026-02-27 19:33:55 -08:00
test-skill
base_audio_transcription_unit_tests.py
base_embedding_unit_tests.py
base_llm_unit_tests.py
base_rerank_unit_tests.py
conftest.py
dog.wav
duck.png
gettysburg.wav
guinea.png
log.xt
Readme.md
test_a2a.py [Feat] Allow calling A2A agents through LiteLLM /chat/completions API (#20358) 2026-02-03 12:52:33 -08:00
test_anthropic_completion.py Fix sonnet 3.7 tests 2026-02-20 17:29:16 -08:00
test_aws_base_llm.py
test_azure_agents.py [Fixes] A2a Gateway - ensure azure foundry agents work (#17943) 2025-12-13 16:08:03 -08:00
test_azure_ai.py [Feat] v2 - Logs view with side panel and improved UX (#20091) 2026-01-30 18:34:13 -08:00
test_azure_o_series.py TestAzureOpenAIO3Mini 2025-12-20 23:18:19 +05:30
test_azure_openai.py fix: support prompt_cache_key for OpenAI and Azure chat completions (#20989) 2026-02-11 12:25:29 -08:00
test_bedrock_agentcore.py fix(agentcore): simplify agentcore streaming (#17141) 2026-01-19 05:20:24 -08:00
test_bedrock_agents.py
test_bedrock_anthropic_regression.py Add Anthropic caching and context tests 2026-02-02 13:25:47 +05:30
test_bedrock_common_utils.py fix(bedrock): strip throughput tier suffixes from model names (#19147) 2026-01-15 18:43:10 -08:00
test_bedrock_completion.py fix: apply custom video pricing from deployment model_info (#21923) 2026-02-23 20:44:02 -08:00
test_bedrock_dynamic_auth_params_unit_tests.py
test_bedrock_embedding.py
test_bedrock_govcloud.py
test_bedrock_gpt_oss.py
test_bedrock_invoke_tests.py [Release Fix] (#22411) 2026-02-28 09:46:35 -08:00
test_bedrock_llama.py
test_bedrock_moonshot.py Add tests for kimi 2 bedrock model 2026-01-08 10:07:33 +05:30
test_bedrock_nova_embedding.py fix(cost-calc): use per-image pricing for Bedrock multimodal embeddings (#21646) 2026-02-20 08:51:21 -08:00
test_bedrock_nova_json.py
test_clarifai_completion.py
test_cloudflare.py
test_cohere.py fix(tests): add flaky retries to flaky CI tests (#21795) 2026-02-21 11:56:29 -08:00
test_containers_api.py [Feat] Containers API - add new container API file management + UI Interface (#17745) 2025-12-09 17:33:26 -08:00
test_convert_dict_to_image.py
test_databricks.py fix: test_databricks_embeddings[True] 2026-01-06 14:20:30 +05:30
test_deepgram.py
test_deepseek_completion.py test_deepseek_mock_completion 2026-01-17 16:36:42 -08:00
test_elevenlabs.py
test_evals_api.py fix(tests): add flaky retries to flaky CI tests (#21795) 2026-02-21 11:56:29 -08:00
test_fireworks_ai_translation.py
test_gemini_image_usage.py fix: ci test 2026-01-17 09:17:31 +09:00
test_gemini.py fix(tests): mock test_claude_tool_use_with_gemini to fix flaky CI (#21832) 2026-02-21 14:34:54 -08:00
test_gigachat.py bugfix: Remove user messages merging 2026-02-03 12:58:28 +00:00
test_gpt4o_audio.py
test_groq.py Add grok reasoning content 2026-01-27 16:34:57 +05:30
test_hosted_vllm_embedding_e2e.py move e2e to llm translation 2026-02-17 22:33:13 +05:30
test_huggingface_chat_completion.py
test_hyperbolic.py
test_infinity.py
test_jina_ai.py
test_lambda_ai.py
test_langgraph.py [Feat] New Provider - add langgraph (#17783) 2025-12-10 12:30:35 -08:00
test_litellm_proxy_provider.py
test_minimax_tts.py Add support for minmax tts 2025-12-23 21:53:53 +05:30
test_mistral_api.py
test_model_cost_map_resilience.py [Stability] Investigate + fix issue where model cost map became poorly formatted (#20895) 2026-02-10 15:17:01 -08:00
test_morph.py
test_nvidia_nim.py TestNvidiaNim 2025-12-13 16:38:11 -08:00
test_openai_o1.py
test_openai.py test_openai_codex_stream 2026-02-14 12:28:18 -08:00
test_openrouter.py fix: ci test 2026-01-17 09:17:31 +09:00
test_optional_params.py Fix: add stop param as supported for openai and azure 2026-02-19 11:26:38 +05:30
test_perplexity_reasoning.py
test_prompt_caching.py
test_prompt_factory.py Preserve forwarding server side called tools 2026-02-27 11:31:45 +05:30
test_replicate.py Add all replicate models in model cost map 2026-01-12 15:19:49 +05:30
test_rerank.py
test_router_llm_translation_tests.py fix(tests): move test_router_azure_acompletion to llm_translation testing (#21837) 2026-02-21 14:41:53 -08:00
test_sambanova_chat_transformation.py
test_skills_api.py
test_skills_e2e.py test_slack_gif_skill_creates_gif 2025-12-20 21:35:25 +05:30
test_snowflake.py
test_text_completion_unit_tests.py fix(openai): use optimized async http client for text completions (#17831) 2025-12-11 14:54:26 -08:00
test_text_completion.py
test_together_ai.py
test_triton.py
test_unit_test_bedrock_invoke.py
test_v0.py
test_voyage_ai.py
test_watsonx.py chore: allow passing scope id for watsonx inferencing (#18959) 2026-01-14 03:47:20 +05:30
test_xai.py Fix usage in xai 2026-02-19 18:48:30 +05:30

Unit tests for individual LLM providers.

Name of the test file is the name of the LLM provider - e.g. test_openai.py is for OpenAI.