litellm/tests
Mateo Wang 51769a8ede
feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support (#29798)
* feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support

Adds a FalAINanoBananaConfig for fal.ai's Nano Banana models, exposed under
both fal-ai/nano-banana and fal-ai/gemini-25-flash-image (identical schema).
This is the migration path for fal-ai/imagen4, which fal deprecates on
2026-06-30.

The config derives the request endpoint from the model name so both aliases
route correctly, maps OpenAI image params to the fal schema (n -> num_images,
size -> nearest supported aspect_ratio, response_format ignored since the model
returns URLs), and reuses the base fal response parser. Pricing is registered
at 0.039 per image in the cost map and backup.

* fix(fal_ai): tighten nano-banana routing and guard mapped params

Match the specific gemini-25-flash-image / gemini-2.5-flash-image
aliases instead of any model containing gemini so future fal.ai
Gemini-branded models aren't silently misrouted to the nano-banana
config. Guard the param mapping on the fal-side keys (num_images,
aspect_ratio) so a pre-set mapped value is respected and an OpenAI
key is never forwarded unmapped.

* fix(fal_ai): drop non-existent gemini-2.5-flash-image routing alias

fal.ai only serves the dotted-free fal-ai/gemini-25-flash-image and
fal-ai/nano-banana endpoints. Routing the dotted gemini-2.5-flash-image
alias built a https://fal.run/fal-ai/gemini-2.5-flash-image URL that
fal.ai 404s and had no pricing entry, so spend tracking silently fell to
zero. Match only the two real endpoint slugs.
2026-06-06 11:16:44 -07:00
..
agent_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests
batches_tests test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
benchmarks
code_coverage_tests Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
documentation_tests
enterprise fix(bedrock): support tool search results + chat annotations (#29120) 2026-05-29 20:48:36 -07:00
guardrails_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
image_gen_tests feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support (#29798) 2026-06-06 11:16:44 -07:00
integration CI: copy of #25177 (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) (#28223) 2026-05-23 12:15:41 -07:00
litellm fix(gemini): googleSearch + server-side tools and googleMaps JSON schema (#29582) 2026-06-04 07:43:30 -07:00
litellm_core_utils
litellm_utils_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
litellm-proxy-extras
llm_responses_api_testing test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview (#29433) 2026-06-01 09:54:30 -07:00
llm_translation test(ci): make the image-gen record/replay proxy report cache mode and per-request HIT/MISS (#29802) 2026-06-06 09:36:06 -07:00
load_tests
local_testing Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
logging_callback_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
mcp_tests [internal copy of #28008] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356) 2026-06-02 12:22:04 -07:00
multi_instance_e2e_tests
ocr_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
old_proxy_tests/tests
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests feat(prometheus): add user_email and user_alias to user budget metrics (#28155) 2026-05-18 16:28:14 -07:00
pass_through_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
pass_through_unit_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
proxy_admin_ui_tests fix(guardrails): persist disable_global_guardrails on keys (#29233) 2026-05-28 21:19:04 -07:00
proxy_behavior test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints (#28681) 2026-05-23 12:16:29 -07:00
proxy_e2e_anthropic_messages_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
proxy_migration_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests Litellm jwt mapping virtualkeys (#28510) 2026-06-04 19:00:36 -07:00
router_unit_tests fix(router): enforce deployment budgets for dynamically added models (#29273) 2026-05-29 19:43:14 -07:00
scim_tests
search_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests
test_litellm feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support (#29798) 2026-06-06 11:16:44 -07:00
unified_google_tests test(google): add google-genai SDK proxy integration tests (#29781) 2026-06-05 21:05:32 +00:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_flush_vcr_cache.py
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py test(ci): make the image-gen record/replay proxy report cache mode and per-request HIT/MISS (#29802) 2026-06-06 09:36:06 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
eval_swe_bench.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py fix(ci): keep coverage rename green when a parallel node runs no tests (#29608) 2026-06-03 13:37:53 -07:00
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team_logging.py
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.