litellm/.circleci
Mateo Wang fdaa288607
ci(circleci): enable Rerun Failed Tests for all pytest jobs (#27155)
* ci(circleci): enable Rerun Failed Tests for all pytest suites

Migrated every pytest-based CircleCI job that uploads JUnit results to use
'circleci tests run' instead of invoking pytest directly. This is the
prerequisite for CircleCI's 'Rerun failed tests' feature to be available
on each job in the pipeline.

For each job:
- Glob test files via 'circleci tests glob' and pipe them into
  'circleci tests run --command="xargs ... pytest ..."' so the agent can
  feed the failed-test subset on rerun.
- Preserve all original pytest flags (parallelism, timeouts, retries,
  coverage, junit output paths).
- For jobs that previously lacked 'store_test_results' (proxy spend
  accuracy, proxy_build_from_pip, db_migration_disable_update_check),
  add the step so JUnit XML is uploaded and rerun is actually wired up.
- Replace the dynamic IGNORE_DIRS shell array in llm_translation_testing
  with a 'grep -v' filter on the glob output, matching the previous
  behavior of skipping tests/llm_translation/realtime.
- For 'build_and_test', glob 'tests/test_*.py' (top-level only) which
  matches the prior 'tests/*.py' shell glob; the long list of
  '--ignore=tests/<subdir>' flags was vestigial and is dropped.

Jobs already using 'circleci tests run' (local_testing_part1/2,
litellm_router_testing) are unchanged.

* fix(ci): convert classnames to file paths on rerun

CircleCI's Rerun Failed Tests sends each previously failed test as a
JUnit classname (e.g. 'tests.otel_tests.test_key_logging_callbacks'),
but pytest needs a file path. Without the awk preprocess step, rerun
runs fail with 'file or directory not found'.

Mirror the awk transform that local_testing_part1, local_testing_part2,
and litellm_router_testing already use, so rerun works in every job
that this PR migrated to 'circleci tests run'.

* ci: drop -x from OTEL pytest run so all failures are reported

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-05-05 17:27:09 -07:00
..
config.yml ci(circleci): enable Rerun Failed Tests for all pytest jobs (#27155) 2026-05-05 17:27:09 -07:00