* ci: add manually-triggered mutation testing smoke workflow
Adds a workflow_dispatch-only GitHub Actions workflow that runs mutmut
against a single source/test pair (router_settings_endpoints) to validate
the tooling end-to-end before scaling.
The workflow reinstalls litellm non-editable so the mutants/ sandbox is
not shadowed by the editable .pth on sys.path, and sets PYTHONPATH so
the trampolined sandbox copy wins over site-packages.
mutmut itself is pulled in via uv run --with so it does not appear in
uv.lock or affect the shared dev environment.
Includes a temporary push: trigger scoped to this branch so we can
iterate before the workflow file lands on the default branch — to be
removed before merging (workflow_dispatch only requires the file on the
default branch to surface the manual trigger button).
* ci(mutation): disable rerun and xdist plugins for mutmut runs
mutmut's in-process pytest.main() call hits
`INTERNALERROR: no option named 'filtered_exceptions'` from
pytest-retry's pytest_configure hook. Reruns are also wrong for
mutation testing — a "failed" mutant test that gets retried would
mask which mutants are killed vs. survive. Disable retry,
rerunfailures, and xdist via pytest_add_cli_args in [tool.mutmut].
* ci(mutation): uninstall pytest-retry before mutmut runs
`-p no:retry` (and similar names) didn't match pytest-retry's
entry-point name, so the plugin still loaded and crashed during
mutmut's "Running clean tests" phase. Uninstalling the package is
surgical and doesn't depend on guessing the entry-point name.
* ci(mutation): emit per-survivor diffs to run-page summary + artifact
The previous artifact only contained `mutmut results` text (which in
mutmut 3.x lists survivor names but not the actual mutations). Adds:
- `mutmut export-cicd-stats` to produce mutmut-cicd-stats.json with the
killed/survived/total scoreboard.
- `mutmut show <name>` per surviving mutant to capture each mutation as
a unified diff.
- A `mutmut-report.md` that combines summary + run-progress tail +
per-survivor diffs, written to both the artifact and
$GITHUB_STEP_SUMMARY (visible on the run page, no download needed).
- Corrected artifact paths: stats files live under mutants/, not the
project root.
- The trampolined source file from the sandbox so survivors can be
inspected even outside `mutmut show`.
* ci(mutation): document intended manual weekly cadence in trigger comment
* ci(mutation): generate ACH-style report with embedded function bodies
Replaces the inline bash markdown generation with a Python script that:
- Groups survivors by function (one section per function, function body
shown once per section, surviving mutants nested as subsections)
- Embeds each enclosing function's source via Python AST (so the agent
has full context, not just a 3-line `mutmut show` diff)
- Inlines the existing test file(s) listed in [tool.mutmut].tests_dir
- Writes an ACH-style task description at the bottom following the
prompt template from arXiv 2501.12862
Output goes to mutation-report.md (artifact) and the head of the file
is appended to $GITHUB_STEP_SUMMARY for at-a-glance visibility.
* fix(mutation report): correctly parse function names with leading underscores
mutmut's mutant-name prefix is x_ (single underscore), so a function
named _foo produces mutants x__foo__mutmut_N. The previous regex
\.x__(.+)__mutmut_ ate the function's leading underscore as part of
the prefix. Changed to \.x_(.+)__mutmut_ so leading underscores are
preserved in the captured function name; verified for normal, leading-
underscore, and dunder-method names.
* feat(mutation report): full Meta ACH-style rendering with MUTANT delimiters
For each surviving mutant, parse the mutmut sandbox trampoline file and
render the mutated function as it appears in the source — with the
differing lines wrapped in `# MUTANT START` / `# MUTANT END` comments,
matching the format from Meta's ACH paper (arXiv 2501.12862, Table 1).
Renames the function header back to its original name so the agent sees
the function as it would appear in the file. Falls back to the unified
diff if the trampoline lookup fails.
Handles replace, insert, and delete diff ops; uses difflib's
SequenceMatcher to find the differing line ranges.
The unified diff is preserved in a collapsible <details> block as
secondary context.
* ci(mutation): scope to whole management_endpoints folder, drop temp push trigger
Final scope before merge:
- paths_to_mutate / tests_dir broadened from one file to the entire
management_endpoints source/test folders
- Trigger is now `workflow_dispatch` only — the temporary push: block
used during workflow iteration is removed
- timeout-minutes bumped from 60 to 350 (just under the GH-hosted job
cap of 360); whole-folder mutation against ~15 files / ~7.5k LOC can
take a few hours
- Artifact path for the trampoline files glob-expanded to cover all
files under mutants/litellm/proxy/management_endpoints/
* fix(mutation report): warn when multiple functions in a file share a name
Addresses the Greptile review concern: ast.walk's first-match-wins
behavior could embed the wrong function body when a file defines the
same name in multiple places (e.g., a module-level helper and a class
method). mutmut's mutant identifier does not carry class context, so
we can't always determine which definition was mutated.
find_function_in_file now returns the start line of every matching
definition; render() surfaces a "Note: N functions named X" warning
in the report when there is more than one match. The first match is
still embedded as the body — the warning tells the reader to verify
manually instead of silently using the wrong context.
Smoke-tested against the existing artifact: single-match files render
unchanged.
* Fix mutation report anchors
* Fix mutation report TOC anchors
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
be84d5cd7d