mirror of
https://github.com/ankitects/anki.git
synced 2026-06-10 07:22:47 -04:00
c1b824258e
## Linked issue Refs #4838 #4839 #4840 ## Summary / motivation Adds a contributor-facing guide for the test coverage setup introduced across the three coverage PRs, and updates CLAUDE.md to reflect that `just` is now the single entry point for all build, test, lint, and format commands. ## How to test - Read `docs/testing-coverage.md` — verify it covers all three stacks, thresholds, and the known gaps section. - Run `just docs` and open `out/docs/html/index.html` to confirm `testing-coverage` appears in the sidebar under Contributing. - Read `CLAUDE.md` — verify `./ninja` / `./tools` references are gone and `just test-rust`, `just test-py`, `just test-ts` are mentioned. ### Details - `docs/testing-coverage.md`: documents tools, thresholds, per-stack entry points, and known gaps for all three coverage stacks. - `docs/index.md`: wired `testing-coverage` into the Sphinx toctree after `contributing`. - `CLAUDE.md`: replaced raw `./ninja` / `./tools` invocations with `just` equivalents; added top-level note directing contributors to `just --list`; added `just test-rust`, `just test-py`, `just test-ts` to the Quick iteration section. --------- Co-authored-by: Abdo <abdo@abdnh.net>
68 lines
3.1 KiB
Markdown
68 lines
3.1 KiB
Markdown
# Testing and Coverage
|
|
|
|
CI runs Rust, Python, and TypeScript tests plus lint/type checks. Coverage is
|
|
orchestrated in `justfile` with direct CLI calls, using ninja only to prepare
|
|
generated build artifacts that the test commands need.
|
|
|
|
## Quick reference
|
|
|
|
```sh
|
|
just test # run all tests (no coverage)
|
|
just test --coverage # run all tests + enforce coverage thresholds
|
|
just test --coverage --html # same + generate HTML reports under out/coverage/
|
|
|
|
just test-rust # Rust only
|
|
just test-rust --coverage
|
|
just test-rust --coverage --html
|
|
|
|
just test-py # Python (pylib + qt) only
|
|
just test-py --coverage
|
|
just test-py --coverage --html
|
|
|
|
just test-ts # TypeScript/Svelte Vitest only
|
|
just test-ts --coverage
|
|
just test-ts --coverage --html
|
|
```
|
|
|
|
HTML reports are written under `out/coverage/` (gitignored).
|
|
|
|
## Coverage tools and thresholds
|
|
|
|
| Stack | Test runner | Coverage tool | Minimum |
|
|
| ------------------- | ------------------------------------ | ---------------- | ------: |
|
|
| Rust workspace | `cargo nextest` via `cargo-llvm-cov` | `cargo-llvm-cov` | 60% |
|
|
| Python `pylib/anki` | `pytest pylib/tests` | `coverage.py` | 65% |
|
|
| Python `qt/aqt` | `pytest qt/tests` | `coverage.py` | 20% |
|
|
| TypeScript/Svelte | `vitest run` | Vitest V8 | 5% |
|
|
|
|
Linux pull requests run `just test --coverage` in CI. macOS and Windows
|
|
jobs run `just test` (no coverage enforcement) for now.
|
|
|
|
## Notes
|
|
|
|
- **Rust** — `cargo-llvm-cov` is installed on demand into `out/bin/` to avoid
|
|
polluting the global cargo install. Coverage runs rebuild the workspace with
|
|
instrumentation, so they are slower than plain `just test-rust`.
|
|
**Windows ARM64 (`aarch64-pc-windows-msvc`) is not supported**: the Rust
|
|
compiler produces malformed `.profraw` files on that target
|
|
([rust-lang/rust#150123](https://github.com/rust-lang/rust/issues/150123),
|
|
[cargo-llvm-cov#436](https://github.com/taiki-e/cargo-llvm-cov/issues/436)).
|
|
`just test-rust --coverage` exits with a clear message on ARM64 Windows;
|
|
use `just test-rust` (no coverage) or rely on CI (Linux) for enforcement.
|
|
- **Python** — coverage is split across two suites (`pylib` and `qt`) because
|
|
they have different `PYTHONPATH` setups and test folders.
|
|
- **TypeScript** — coverage is measured only over code reachable through
|
|
Vitest's module graph. Svelte component rendering behavior is not covered.
|
|
|
|
## Gaps and future improvements
|
|
|
|
- Raise thresholds gradually as the test suite grows and CI timings stabilise.
|
|
- Exclude generated files from coverage denominators where appropriate.
|
|
- Publish `out/coverage/` as a CI artifact so reviewers can browse HTML
|
|
reports directly from a PR.
|
|
- Consider diff/changed-file coverage once baselines are stable — it is a
|
|
better enforcement mechanism for incremental improvement than whole-repo
|
|
thresholds.
|
|
- Add component or browser tests for Svelte UI surfaces if Svelte coverage
|
|
is intended to cover rendered component behaviour.
|