ai-workspace-services/litellm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Fix] Scope documentation workflow to match CircleCI and add missing router settings

Browse Source 
Revert path fixes for documentation tests that CircleCI never ran
(test_exception_types, test_general_setting_keys, test_readme_providers,
test_standard_logging_payload). Update the GHA workflow to run only the
4 tests CircleCI actually executed: test_env_keys, test_router_settings,
test_api_docs, test_circular_imports.

Add 2 missing router_settings keys (enable_health_check_routing,
health_check_staleness_threshold) and 27 missing general_settings keys
to config_settings.md so test_router_settings passes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yuneng Jiang 2026-03-28 11:23:53 -07:00
parent 7100ed5d0a
commit 7851567091
No known key found for this signature in database
6 changed files with 91 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

57
.github/workflows/test-unit-documentation.yml vendored
View File

@ -13,8 +13,55 @@ concurrency:
jobs:
documentation:
uses: ./.github/workflows/_test-unit-base.yml
with:
test-path: "tests/documentation_tests"
workers: 2
reruns: 1
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Install Poetry
run: pip install 'poetry==2.3.2'
- name: Cache Poetry dependencies
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cache/pypoetry
~/.cache/pip
.venv
key: ${{ runner.os }}-poetry-${{ hashFiles('poetry.lock') }}
restore-keys: |
${{ runner.os }}-poetry-
- name: Install dependencies
run: |
poetry config virtualenvs.in-project true
poetry install --with dev,proxy-dev --extras "proxy semantic-router"
poetry run pip install google-genai==1.22.0 \
google-cloud-aiplatform==1.115.0 fastapi-offline==1.7.3 python-multipart==0.0.22 openapi-core==0.23.0
- name: Setup litellm-enterprise
run: |
poetry run pip install --force-reinstall --no-deps -e enterprise/
- name: Generate Prisma client
env:
PRISMA_BINARY_CACHE_DIR: ${{ runner.temp }}/prisma-cache
run: |
poetry run pip install nodejs-wheel-binaries==24.13.1
poetry run prisma generate --schema litellm/proxy/schema.prisma
# Run the same documentation tests that CircleCI ran (as direct Python scripts)
- name: Run documentation validation tests
run: |
poetry run python ./tests/documentation_tests/test_env_keys.py
poetry run python ./tests/documentation_tests/test_router_settings.py
poetry run python ./tests/documentation_tests/test_api_docs.py
poetry run python ./tests/documentation_tests/test_circular_imports.py

29
docs/my-website/docs/proxy/config_settings.md
View File

@ -279,6 +279,33 @@ router_settings:
| forward_client_headers_to_llm_api | boolean | If true, forwards the client headers (any `x-` headers and `anthropic-beta` headers) to the backend LLM call |
| maximum_spend_logs_retention_period | str | Used to set the max retention time for spend logs in the db, after which they will be auto-purged |
| maximum_spend_logs_retention_interval | str | Used to set the interval in which the spend log cleanup task should run in. |
| alert_type_config | dict | Configuration mapping alert types to their handler settings |
| always_include_stream_usage | boolean | If true, includes usage metrics in every streaming response chunk |
| auto_redirect_ui_login_to_sso | boolean | If true, automatically redirects UI login page to SSO provider |
| control_plane_url | string | URL of the control plane for cross-instance state sharing |
| custom_auth_run_common_checks | boolean | If true, runs standard auth validation checks alongside custom auth handlers |
| custom_ui_sso_sign_in_handler | string | Custom handler for SSO sign-in logic in the UI |
| database_connection_pool_timeout | integer | Database connection pool timeout in seconds |
| disable_error_logs | boolean | If true, suppresses error tracking and storage in the database |
| enable_health_check_routing | boolean | If true, enables health check-driven request routing to avoid unhealthy deployments |
| enable_mcp_registry | boolean | If true, enables access to the centralized MCP server registry |
| enforce_rbac | boolean | If true, enables role-based access control (RBAC) for all proxy operations |
| forward_llm_provider_auth_headers | boolean | If true, forwards provider-specific auth headers to LLM API calls |
| health_check_concurrency | integer | Maximum number of concurrent health check operations |
| health_check_staleness_threshold | integer | Maximum age in seconds for health check results before marking deployments as stale |
| maximum_spend_logs_cleanup_cron | string | Cron expression for scheduling automatic spend log cleanup tasks |
| mcp_client_side_auth_header_name | string | HTTP header name for client-side MCP server credentials |
| mcp_internal_ip_ranges | list | CIDR ranges considered internal for non-public MCP server access control |
| mcp_required_fields | list | List of required field names for MCP server submissions |
| mcp_trusted_proxy_ranges | list | CIDR ranges of proxies trusted to forward X-Forwarded-For headers for MCP |
| require_end_user_mcp_access_defined | boolean | If true, requires end users to have explicit MCP access permissions defined |
| role_permissions | list | List of role-based permission configurations |
| search_tools | list | List of search tool configurations for enabling web search capabilities |
| token_rate_limit_type | string | Rate limit counting method: "total", "output", or "input" tokens |
| use_redis_transaction_buffer | boolean | If true, buffers database transactions in Redis before writing |
| use_shared_health_check | boolean | If true, uses Redis-backed shared health check state across multiple proxy instances |
| user_header_mappings | dict | Map custom request headers to user IDs using lookup rules |
| user_header_name | string | HTTP header name to extract user identity from requests |
### router_settings - Reference
@ -367,6 +394,8 @@ router_settings:
| ignore_invalid_deployments | boolean | If true, ignores invalid deployments. Default for proxy is True - to prevent invalid models from blocking other models from being loaded. |
| search_tools | List[SearchToolTypedDict] | List of search tool configurations for Search API integration. Each tool specifies a search_tool_name and litellm_params with search_provider, api_key, api_base, etc. [Further Docs](../search/index.md) |
| guardrail_list | List[GuardrailTypedDict] | List of guardrail configurations for guardrail load balancing. Enables load balancing across multiple guardrail deployments with the same guardrail_name. [Further Docs](./guardrails/guardrail_load_balancing.md) |
| enable_health_check_routing | boolean | If true, enables health check-driven deployment filtering to avoid routing requests to unhealthy deployments |
| health_check_staleness_threshold | integer | Maximum age in seconds for cached health check results before marking deployments as stale |
### environment variables - Reference

8
tests/documentation_tests/test_exception_types.py
View File

@ -32,12 +32,10 @@ error_names = {
# Parse the documentation to extract documented keys
_test_dir = os.path.dirname(os.path.abspath(__file__))
repo_base = os.path.abspath(os.path.join(_test_dir, "..", ".."))
# repo_base = "./"
repo_base = "../../"
print(os.listdir(repo_base))
docs_path = os.path.join(
repo_base, "docs", "my-website", "docs", "exception_mapping.md"
)
docs_path = f"{repo_base}/docs/my-website/docs/exception_mapping.md" # Path to the documentation
documented_keys = set()
try:
with open(docs_path, "r", encoding="utf-8") as docs_file:

11
tests/documentation_tests/test_general_setting_keys.py
View File

@ -2,9 +2,7 @@ import os
import re
# Define the base directory for the litellm repository and documentation path
_test_dir = os.path.dirname(os.path.abspath(__file__))
_repo_root = os.path.abspath(os.path.join(_test_dir, "..", ".."))
repo_base = os.path.join(_repo_root, "litellm")
repo_base = "./litellm" # Change this to your actual path
# Regular expressions to capture the keys used in general_settings.get() and general_settings[]
@ -34,9 +32,10 @@ for root, dirs, files in os.walk(repo_base):
general_settings_keys.update(bracket_matches)
# Parse the documentation to extract documented keys
print(os.listdir(_repo_root))
docs_path = os.path.join(
_repo_root, "docs", "my-website", "docs", "proxy", "config_settings.md"
repo_base = "./"
print(os.listdir(repo_base))
docs_path = (
"./docs/my-website/docs/proxy/config_settings.md" # Path to the documentation
)
documented_keys = set()
try:

4
tests/documentation_tests/test_readme_providers.py
View File

@ -7,9 +7,7 @@ import re
from litellm.types.utils import LlmProviders
# Define paths
_test_dir = os.path.dirname(os.path.abspath(__file__))
_repo_root = os.path.abspath(os.path.join(_test_dir, "..", ".."))
readme_path = os.path.join(_repo_root, "README.md")
readme_path = "./README.md"
# Providers that shouldn't be required in README
# (specialized tools, observability, database providers that aren't LLM providers)

6
tests/documentation_tests/test_standard_logging_payload.py
View File

@ -38,11 +38,7 @@ def test_standard_logging_payload_documentation():
print(_field)
# Read the documentation
_test_dir = os.path.dirname(os.path.abspath(__file__))
_repo_root = os.path.abspath(os.path.join(_test_dir, "..", ".."))
docs_path = os.path.join(
_repo_root, "docs", "my-website", "docs", "proxy", "logging_spec.md"
)
docs_path = "../../docs/my-website/docs/proxy/logging_spec.md"
try:
with open(docs_path, "r", encoding="utf-8") as docs_file: