anki/docs/testing-coverage.md

Testing and Coverage

CI runs Rust, Python, and TypeScript tests plus lint/type checks. Coverage is orchestrated in justfile with direct CLI calls, using ninja only to prepare generated build artifacts that the test commands need.

Quick reference

just test                        # run all tests (no coverage)
just test --coverage             # run all tests + enforce coverage thresholds
just test --coverage --html      # same + generate HTML reports under out/coverage/

just test-rust                   # Rust only
just test-rust --coverage
just test-rust --coverage --html

just test-py                     # Python (pylib + qt) only
just test-py --coverage
just test-py --coverage --html

just test-ts                     # TypeScript/Svelte Vitest only
just test-ts --coverage
just test-ts --coverage --html

HTML reports are written under out/coverage/ (gitignored).

Coverage tools and thresholds

Stack	Test runner	Coverage tool	Minimum
Rust workspace	cargo nextest via cargo-llvm-cov	cargo-llvm-cov	60%
Python pylib/anki	pytest pylib/tests	coverage.py	65%
Python qt/aqt	pytest qt/tests	coverage.py	20%
TypeScript/Svelte	vitest run	Vitest V8	5%

Linux pull requests run just test --coverage in CI. macOS and Windows jobs run just test (no coverage enforcement) for now.

Notes

• Rust — cargo-llvm-cov is installed on demand into out/bin/ to avoid polluting the global cargo install. Coverage runs rebuild the workspace with instrumentation, so they are slower than plain just test-rust. Windows ARM64 (aarch64-pc-windows-msvc) is not supported: the Rust compiler produces malformed .profraw files on that target (rust-lang/rust#150123, cargo-llvm-cov#436). just test-rust --coverage exits with a clear message on ARM64 Windows; use just test-rust (no coverage) or rely on CI (Linux) for enforcement.
• Python — coverage is split across two suites (pylib and qt) because they have different PYTHONPATH setups and test folders.
• TypeScript — coverage is measured only over code reachable through Vitest's module graph. Svelte component rendering behavior is not covered.

Gaps and future improvements

• Raise thresholds gradually as the test suite grows and CI timings stabilise.
• Exclude generated files from coverage denominators where appropriate.
• Publish out/coverage/ as a CI artifact so reviewers can browse HTML reports directly from a PR.
• Consider diff/changed-file coverage once baselines are stable — it is a better enforcement mechanism for incremental improvement than whole-repo thresholds.
• Add component or browser tests for Svelte UI surfaces if Svelte coverage is intended to cover rendered component behaviour.