ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
e5bbafba62
litellm/tests/proxy_unit_tests
History
Achilleas Athanasiou Fragkoulis cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 
Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
..
example_config_yaml
test_configs
test_model_response_typing
adroit-crow-413218-bc47f303efc9.json
azure_fine_tune.jsonl
batch_job_results_furniture.jsonl
conftest copy.py
conftest.py
data_map.txt
eagle.wav
gettysburg.wav
large_text.py
messages_with_counts.py
model_cost.json
openai_batch_completions_router.jsonl
openai_batch_completions.jsonl
speech_vertex.mp3
test_aproxy_startup.py
test_audit_logs_proxy.py
test_auth_checks.py Fixing tests 2026-02-05 21:44:00 -08:00
test_banned_keyword_list.py
test_check_responses_cost.py Add cost tracking for responses api in background mode 2025-12-19 13:35:48 +05:30
test_custom_callback_input.py
test_custom_logger_s3_gcs.py
test_custom_tokenizer_bug.py
test_db_schema_changes.py
test_db_schema_migration.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_default_end_user_budget_simple.py
test_deployed_proxy_keygen.py
test_e2e_pod_lock_manager.py
test_get_image.py fix: optimize logo fetching and resolve mcp import blockers (#19719) 2026-01-25 23:08:16 -08:00
test_google_endpoint_routing.py Support model names with slashes on Gemini endpoints (#17743) 2025-12-09 18:40:51 -08:00
test_google_gemini_proxy_request.py
test_jwt.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_key_generate_dynamodb.py
test_key_generate_prisma.py fixing test 2026-01-16 19:06:23 -08:00
test_models_fallback_endpoint.py
test_prisma_client_backoff_retry.py
test_prompt_test_endpoint.py [Feat] UI - Prompt Management - Allow testing prompts with Chat UI (#16898) 2025-11-21 08:53:18 -08:00
test_proxy_config_unit_test.py fix: enable JSON logging via configuration and add regression test 2026-01-13 09:38:19 -07:00
test_proxy_custom_auth.py
test_proxy_custom_logger.py test_embedding fix 2026-01-31 13:43:53 -08:00
test_proxy_encrypt_decrypt.py
test_proxy_exception_mapping.py [Perf] Embeddings: Use router's O(1) lookup and shared sessions (#16344) 2025-11-14 09:21:45 -08:00
test_proxy_gunicorn.py
test_proxy_pass_user_config.py
test_proxy_reject_logging.py
test_proxy_routes.py test: handle wildcard routes in route validation test 2026-01-16 12:53:27 +09:00
test_proxy_server_caching.py
test_proxy_server_keys.py
test_proxy_server_langfuse.py
test_proxy_server_spend.py
test_proxy_server.py Fixing tests 2026-01-26 20:07:30 -08:00
test_proxy_setting_guardrails.py
test_proxy_token_counter.py fix mock_handler 2026-01-20 18:01:22 -08:00
test_proxy_utils.py Fix test_provider_specific_header_in_request 2026-02-11 16:56:26 +05:30
test_realtime_cache.py [Perf] Fix bottlenecks degrading realtime endpoint performance (#16670) 2025-11-22 10:01:02 -08:00
test_response_polling_handler.py Add tests for native_background_mode feature 2026-01-27 16:48:22 -08:00
test_search_api_logging.py Add cost tracking and logging support 2025-11-25 17:14:59 +05:30
test_server_root_path.py fix: server rooth path (#19790) 2026-01-26 09:48:06 -08:00
test_skills_db.py test_create_skill_sdk 2025-12-20 15:08:31 +05:30
test_ui_path_detection.py fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492) 2026-02-12 19:39:04 +05:30
test_unit_test_max_model_budget_limiter.py fix conflicts with main- (this PR is from upstream/main) 2026-02-07 03:10:53 +05:30
test_unit_test_proxy_hooks.py [Fix] CI/CD – Clean Up Performance PR Changes & others (#17838) 2025-12-11 12:50:03 -08:00
test_update_spend.py [Fix] CI/CD – Clean Up Performance PR Changes & others (#17838) 2025-12-11 12:50:03 -08:00
test_user_api_key_auth.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_zero_cost_model_budget_bypass.py Add support for 0 cost models 2026-02-02 13:29:37 +05:30
vertex_key.json