Mutation testing
Unit tests pass is not proof they check anything. Mutation testing changes the code (a <
into <=, a branch into true) and expects some test to fail; a mutant that survives
marks code no test watches. Fix the test, not the config.
Run the tools one at a time — each compiles and runs the suite hundreds of times.
| Layer | Tool | Command | Scope / config | Report |
|---|---|---|---|---|
| Rust | cargo-mutants | pnpm mutants:rust | .cargo/mutants.toml | mutants.out/ |
| TypeScript | Stryker + vitest | pnpm mutants:js | stryker.config.json (mutate) | reports/mutation/mutation.html |
| Kotlin | PIT (gradle-pitest-plugin) | pnpm mutants:kotlin | android/pitest.gradle, applied from build.gradle.kts only with -Ppitest=true | android/build/reports/pitest/ |
| Swift | Muter | pnpm mutants:swift | swift/muter.conf.yml + --files-to-mutate Resolve.swift | swift/muter_logs/ |
Install once: cargo install --locked cargo-mutants, brew install muter-mutation-testing/formulae/muter.
Stryker and vitest come with pnpm install; PIT is fetched by Gradle.
Speed first
A mutation run costs mutants × (rebuild + tests), so slow tests multiply. Before a run:
- Rust:
[profile.dev.package."*"] opt-level = 2optimizes dependencies only (the resvg raster tests went 4.1 s → 0.8 s), and[profile.mutants]drops debug info (rebuild per mutant 3.2 s → 2.4 s). Find slow tests with the loop in the commit that added this page orcargo test -- -Z unstable-options --report-timeon nightly. - Keep each tool on code its unit tests reach: Stryker mutates
widget-src/resolve.ts, PIT the pure Kotlin classes (Resolve, store keys, scheduler math), Muter onlyResolve.swift(--files-to-mutate). UI code is covered by the golden-image stands instead.
Current scores
| Layer | Result |
|---|---|
| Rust | 5 survivors left, all equivalent: TTL < / <= boundaries, and the image.url ! where the cell is full on every platform |
| TypeScript | 97.6% on widget-src/resolve.ts |
| Kotlin | 91% killed, 99% line coverage of the targeted classes |
| Swift | 96%; the one reported survivor (the bool ternary in Resolve.swift text()) is a Muter false positive — the test fails when that mutation is applied by hand |
Local only
Mutation tools are not run in CI: run them locally, one tool at a time, after the feature
work and the scenario tests are done (pnpm mutants:rust, mutants:js, mutants:kotlin,
mutants:swift). cargo-mutants without -j when other heavy work shares the machine.
Changed lines only: git diff main -- '*.rs' > pr.diff && cargo mutants --in-diff pr.diff.
Shared fixtures
tests/fixtures/resolve/*.json are run by all four ports (src/resolve.rs,
widget-src/resolve.ts, Resolve.swift, Resolve.kt). Add a case there when the
contract changes; every platform then has to agree.