[Fix] Tests: Restore /metrics access for prometheus test suite

/metrics now requires auth by default; tests/otel_tests/test_prometheus.py
makes 4+ unauthenticated GETs against http://0.0.0.0:4000/metrics, so
every prometheus test in CI now fails the metric assertion.

Set require_auth_for_metrics_endpoint: false in otel_test_config.yaml
to opt out for this test job, which scrapes /metrics directly. Verified
locally: 8/8 prometheus tests green (one flaky retry on
test_proxy_success_metrics that pre-dates this PR).

Also drop the -x stop-on-first-failure flag from the otel test command
so all failures in the job surface in a single CI run rather than
hiding behind whichever one trips first.
This commit is contained in:
Yuneng Jiang 2026-05-04 20:54:54 -07:00
parent e6f524f951
commit 0976fbc6c4
No known key found for this signature in database
2 changed files with 4 additions and 1 deletions

View File

@ -1475,7 +1475,7 @@ jobs:
- run:
name: Run tests
command: |
uv run --no-sync python -m pytest -v tests/otel_tests -x --junitxml=test-results/junit.xml --durations=5
uv run --no-sync python -m pytest -v tests/otel_tests --junitxml=test-results/junit.xml --durations=5
no_output_timeout: 15m
# Clean up first container
- run:

View File

@ -46,6 +46,9 @@ litellm_settings:
cache: true
callbacks: ["otel", "prometheus"]
disable_end_user_cost_tracking_prometheus_only: True
# /metrics auth is on by default; tests/otel_tests/test_prometheus.py
# scrapes the endpoint without credentials, so opt out here.
require_auth_for_metrics_endpoint: False
guardrails:
- guardrail_name: "bedrock-pre-guard"