litellm/tests
Mateo Wang b3297fc2ea
feat(proxy): hot-reload .env in dev when running with --reload (#29783)
* feat(proxy): hot-reload .env in dev when running with --reload

The --reload watcher already restarts the worker on *.py and --config YAML
edits, but .env was unwatched, so changing a key there did nothing until a
manual restart. Add .env to the uvicorn reload_includes (and to the
StatReload monkeypatch, which ignores reload_includes) so an edit triggers a
worker restart.

A reloaded worker is a fresh process that inherits the reloader's
environment, so load_dotenv(override=False) would keep serving the stale
inherited value for any key already in the environment. The CLI now exports
LITELLM_DEV_ENV_HOT_RELOAD when --reload is set, and litellm/__init__.py
reads it to load .env with override=True only on that dev path, leaving
normal startup precedence untouched.

* feat(proxy): warn that --reload makes .env override shell env vars

When --reload is active, worker processes re-read .env with override=True, so
.env values win over shell-exported environment variables. Surface this dotenv
precedence change with a startup warning so a developer who relies on a
shell-exported override is not silently surprised.

* fix(proxy): type reload helper paths as Optional[str] to satisfy mypy

* fix(proxy): watch the cwd .env in both reload backends for parity

WatchFiles only watches cwd (and the --config dir) for .env, while the
StatReload fallback used find_dotenv(usecwd=True), which walks up to a
parent-dir .env that WatchFiles never sees. Point StatReload at the same
cwd .env so the two reload backends react to the same file.
2026-06-06 09:39:21 -07:00
..
agent_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests
batches_tests test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
benchmarks
code_coverage_tests Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
documentation_tests
enterprise fix(bedrock): support tool search results + chat annotations (#29120) 2026-05-29 20:48:36 -07:00
guardrails_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
image_gen_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
integration CI: copy of #25177 (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) (#28223) 2026-05-23 12:15:41 -07:00
litellm fix(gemini): googleSearch + server-side tools and googleMaps JSON schema (#29582) 2026-06-04 07:43:30 -07:00
litellm_core_utils
litellm_utils_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
litellm-proxy-extras
llm_responses_api_testing test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview (#29433) 2026-06-01 09:54:30 -07:00
llm_translation test(ci): make the image-gen record/replay proxy report cache mode and per-request HIT/MISS (#29802) 2026-06-06 09:36:06 -07:00
load_tests
local_testing Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
logging_callback_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
mcp_tests [internal copy of #28008] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356) 2026-06-02 12:22:04 -07:00
multi_instance_e2e_tests
ocr_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
old_proxy_tests/tests
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests feat(prometheus): add user_email and user_alias to user budget metrics (#28155) 2026-05-18 16:28:14 -07:00
pass_through_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
pass_through_unit_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
proxy_admin_ui_tests fix(guardrails): persist disable_global_guardrails on keys (#29233) 2026-05-28 21:19:04 -07:00
proxy_behavior test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints (#28681) 2026-05-23 12:16:29 -07:00
proxy_e2e_anthropic_messages_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
proxy_migration_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests Litellm jwt mapping virtualkeys (#28510) 2026-06-04 19:00:36 -07:00
router_unit_tests fix(router): enforce deployment budgets for dynamically added models (#29273) 2026-05-29 19:43:14 -07:00
scim_tests
search_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests
test_litellm feat(proxy): hot-reload .env in dev when running with --reload (#29783) 2026-06-06 09:39:21 -07:00
unified_google_tests test(google): add google-genai SDK proxy integration tests (#29781) 2026-06-05 21:05:32 +00:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_flush_vcr_cache.py
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py test(ci): make the image-gen record/replay proxy report cache mode and per-request HIT/MISS (#29802) 2026-06-06 09:36:06 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
eval_swe_bench.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py fix(ci): keep coverage rename green when a parallel node runs no tests (#29608) 2026-06-03 13:37:53 -07:00
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team_logging.py
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.