mirror of
https://github.com/ankitects/anki.git
synced 2026-09-10 15:48:11 -04:00
a7f3ffb63a
## Linked issue Closes #4995 ## Summary / motivation Bedges available: [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) [](https://sonarcloud.io/summary/new_code?id=ankitects_anki) **[SONAR DASHBOARD](https://sonarcloud.io/project/overview?id=ankitects_anki)** Integrates SonarCloud into the CI pipeline as a proof of concept to evaluate whether it surfaces actionable quality and security insights for this codebase. Changes: - Added `sonar-project.properties` configuring sources (`pylib`, `qt`, `ts`, `rslib`) and coverage report paths - Extended `tools/coverage/coverage-py` (and `.bat`) to emit `coverage.xml` (Cobertura) - Extended `tools/coverage/coverage-ts` (and `.bat`) to emit `lcov.info` via the V8 provider - Extended `tools/coverage/coverage-rust` (and `.bat`) to emit `lcov.info` via `cargo-llvm-cov report` - Added a `SonarCloud Scan` step to the `check-linux` CI job, running after all checks pass and before the build cache is saved ## How to test ### Details **1. Install sonar-scanner** ```bash brew install sonar-scanner ``` **2. Generate and configure a token** Go to [sonarcloud.io](https://sonarcloud.io/) → My Account → Security → Generate Token Copy the generated token and export it in your shell: ``` export SONAR_TOKEN=your_token_here ``` **3. Generate coverage reports** ``` just test --coverage ``` Expected output files: - out/coverage/python-pylib/coverage.xml - out/coverage/python-qt/coverage.xml - out/coverage/typescript/lcov.info - out/coverage/rust/lcov.info **4. Run the scanner manually** ``` sonar-scanner ``` Results will appear in the SonarCloud dashboard To test coverage generation locally: ```bash just coverage # verify files exist: # out/coverage/python-pylib/coverage.xml # out/coverage/python-qt/coverage.xml # out/coverage/typescript/lcov.info # out/coverage/rust/lcov.info ```
91 lines
3.9 KiB
Markdown
91 lines
3.9 KiB
Markdown
<!-- DO NOT MANUALLY EDIT THIS FILE -->
|
|
<!-- This file is copied from docs-site/developers/testing-coverage.mdx automatically -->
|
|
|
|
# Testing and Coverage
|
|
|
|
<!-- <<<cog
|
|
from cogdocs import get_file_contents
|
|
cog.out(get_file_contents("testing-coverage"))
|
|
>>> -->
|
|
|
|
CI runs Rust, Python, and TypeScript tests plus lint/type checks. Coverage is
|
|
orchestrated in `justfile` with direct CLI calls, using ninja only to prepare
|
|
generated build artifacts that the test commands need.
|
|
|
|
## Quick reference
|
|
|
|
```sh
|
|
just test # run all tests (no coverage)
|
|
just test --coverage # run all tests + enforce coverage thresholds
|
|
just test --coverage --html # same + generate HTML reports under out/coverage/
|
|
|
|
just test-rust # Rust only
|
|
just test-rust --coverage
|
|
just test-rust --coverage --html
|
|
|
|
just test-py # Python (pylib + qt) only
|
|
just test-py --coverage
|
|
just test-py --coverage --html
|
|
|
|
just test-ts # TypeScript/Svelte Vitest only
|
|
just test-ts --coverage
|
|
just test-ts --coverage --html
|
|
```
|
|
|
|
HTML reports are written under `out/coverage/` (gitignored).
|
|
|
|
## Coverage tools and thresholds
|
|
|
|
| Stack | Test runner | Coverage tool | Minimum |
|
|
| ------------------- | ------------------------------------ | ---------------- | ------: |
|
|
| Rust workspace | `cargo nextest` via `cargo-llvm-cov` | `cargo-llvm-cov` | 60% |
|
|
| Python `pylib/anki` | `pytest pylib/tests` | `coverage.py` | 63% |
|
|
| Python `qt/aqt` | `pytest qt/tests` | `coverage.py` | 20% |
|
|
| TypeScript/Svelte | `vitest run` | Vitest V8 | 5% |
|
|
|
|
Linux pull requests run `just test --coverage` in CI. macOS and Windows
|
|
jobs run `just test` (no coverage enforcement) for now.
|
|
|
|
## Notes
|
|
|
|
- **Rust** — `cargo-llvm-cov` is installed on demand into `out/bin/` to avoid
|
|
polluting the global cargo install. Coverage runs rebuild the workspace with
|
|
instrumentation, so they are slower than plain `just test-rust`.
|
|
**Windows ARM64 (`aarch64-pc-windows-msvc`) is not supported**: the Rust
|
|
compiler produces malformed `.profraw` files on that target
|
|
([rust-lang/rust#150123](https://github.com/rust-lang/rust/issues/150123),
|
|
[cargo-llvm-cov#436](https://github.com/taiki-e/cargo-llvm-cov/issues/436)).
|
|
`just test-rust --coverage` exits with a clear message on ARM64 Windows;
|
|
use `just test-rust` (no coverage) or rely on CI (Linux) for enforcement.
|
|
- **Python** — coverage is split across two suites (`pylib` and `qt`) because
|
|
they have different `PYTHONPATH` setups and test folders.
|
|
- **TypeScript** — coverage is measured only over code reachable through
|
|
Vitest's module graph. Svelte component rendering behavior is not covered.
|
|
|
|
## SonarCloud
|
|
|
|
SonarCloud runs automatically on every push and pull request via CI. It aggregates
|
|
the coverage reports generated by the three stacks and runs static analysis across
|
|
Python, TypeScript, and Rust.
|
|
|
|
- **Dashboard**: https://sonarcloud.io/project/overview?id=ankitects_anki
|
|
- **Coverage**: uses the same LCOV/XML reports produced by `just test --coverage`
|
|
- **Analysis results**: SonarCloud posts a comment on each PR with quality gate status
|
|
|
|
The quality gate is configured as non-blocking — a failing gate does not prevent
|
|
merging, but the results are visible on the PR.
|
|
|
|
## Gaps and future improvements
|
|
|
|
- Raise thresholds gradually as the test suite grows and CI timings stabilise.
|
|
- Exclude generated files from coverage denominators where appropriate.
|
|
- Publish `out/coverage/` as a CI artifact so reviewers can browse HTML
|
|
reports directly from a PR.
|
|
- Consider diff/changed-file coverage once baselines are stable — it is a
|
|
better enforcement mechanism for incremental improvement than whole-repo
|
|
thresholds.
|
|
- Add component or browser tests for Svelte UI surfaces if Svelte coverage
|
|
is intended to cover rendered component behaviour.
|
|
|
|
<!-- <<<end>>> -->
|