## Linked issue (required)
Fixes#5110
## Summary / motivation (required)
CI builds the fcitx5 plugin against one Qt but ships it inside PyQt6's
Qt — a
version mismatch that crashes Anki on startup for every fcitx5 user.
**Root cause:** the plugin subclasses `QPlatformInputContext`, a Qt
private API
with no ABI guarantee across minor versions. Built against Qt 6.2.4
headers but
run inside Qt 6.11.0, it dereferences a misplaced member and SIGSEGVs on
first
focus.
**Solution:** build the plugin against the same Qt as the bundled
`pyqt6-qt6`
and install it where the packaging step (`qt/tools/build_installer.py`)
looks.
The per-flag reasoning is in the commit message.
## Steps to reproduce (required, use N/A if not applicable)
1. On a Linux desktop configured to use fcitx5 as the input method,
install the
official Anki 26.05 release.
2. Launch Anki normally.
3. It dies with SIGSEGV on startup, as soon as the main window takes
focus.
## How to test (required)
### Checklist (minimum)
- [x] I ran `./ninja check` or an equivalent relevant check locally.
- [ ] I added or updated tests when the change is non-trivial or
behavior changed.
### Details
`release.yml` is `workflow_dispatch`-only, so it can't run on the PR. I
reproduced its build step locally: rebuilt the plugin against Qt 6.11.0
(confirmed the `Qt_6.11` ELF tag and a clean `dlopen(RTLD_NOW)`),
dropped it into
an Anki 26.05 bundle, and typed Chinese and Japanese without the crash.
## Risk / compatibility / migration (optional)
1. Fixes `build-linux-x86` only. `build-linux-arm-installer` installs a
prebuilt
`fcitx5-frontend-qt6` from apt, built against Ubuntu's Qt rather than
the
bundled 6.11.0 — the same class of mismatch. I have no arm hardware to
confirm
or fix it, so it's left for a follow-up.
2. Adds an `aqtinstall` step (~1.5 GB Qt download) to the Linux release
build.
3. **Not validated on CI.** `release.yml` is `workflow_dispatch`-only,
so an
external contributor can't run it; only its build step is reproduced
locally
(see How to test). **Please confirm it in a real release build.**
## UI evidence (required for visual changes; otherwise N/A)
N/A
## Scope
- [x] This PR is focused on one change (no unrelated edits).
## Linked issue (required)
Refs #5134
## Summary / motivation (required)
When running ci, instead of manually compiling cargo-llvm-cov and
cargo-nextest when not cached, this pr pulls them in as prebuilt
binaries instead
## Steps to reproduce (required, use N/A if not applicable)
N/A
## How to test (required)
CI should pass, and running `./check` and `just test --coverage` should
work locally too
### Checklist (minimum)
- [x] I ran `./ninja check` or an equivalent relevant check locally.
- [ ] I added or updated tests when the change is non-trivial or
behavior changed.
### Details
Continuing from
https://github.com/ankitects/anki/issues/5134#issuecomment-4936603431,
we can avoid installing the crates globally when run locally, but in ci
it doesn't really matter so we can use install-action as per normal.
W.r.t nextest's features, all `default-no-update` does is remove its
ability to self-update. Since we're puling in prebuilt binaries during
ci there's no difference
I've left out n2 for now, as without a release workflow on its repo or
on anki's fork, we'd still have to compile it ourselves (caching it can
be unsafe, but it sort of is already cached by setup-rust-toolchain?)
## Scope
- [x] This PR is focused on one change (no unrelated edits).
## Summary / motivation
Two independent gaps in the `check-linked-issue` workflow caused false
positives on valid PRs:
1. **Cross-repo references not recognised** — the regex only matched
bare `#NNN` but GitHub also accepts `owner/repo#NNN`. Any PR referencing
an issue from another repository (e.g. `Refs ankitects/ankimobile#10`)
was still flagged. Observed in #5111.
2. **Org members should be trusted contributors** — maintainers and core
team members already know the process; requiring them to link an issue
on every PR adds friction with no benefit. The check now calls the
GitHub membership API and skips the requirement for org members
entirely.
## Steps to reproduce
1. Open a PR with `Refs ankitects/ankimobile#10` as the linked issue →
bot flags it as missing.
2. Open a PR as an org member without a linked issue → bot should not
flag it.
## How to test
1. Open a draft PR from an external contributor with `Refs
owner/repo#NNN` → verify **no** `missing-issue` label.
2. Open a draft PR as an org member with no linked issue → verify **no**
label or comment.
3. Open a draft PR as an external contributor with no linked issue →
verify label and comment **are** applied.
## Summary / motivation
Two related bugs in the `check-linked-issue` workflow caused unnecessary
noise on PRs:
1. **Wrong keywords**: the regex only accepted `closes`, `fixes`, and
`resolves` variants, but the PR template explicitly shows `Refs #123` as
a valid option. Any PR using `Refs #NNN`, `Ref #NNN`, or `References
#NNN` was incorrectly flagged. Observed in #5087.
2. **Stale bot comment**: once the contributor added an issue link and
the `missing-issue` label was removed, the bot's warning comment
remained visible on the PR, creating unnecessary noise.
## Steps to reproduce (required, use N/A if not applicable)
1. Open a PR with `Refs #<open issue>` in the linked issue section
(following the template hint).
2. Bot applies `missing-issue` label and posts a comment despite the
issue being referenced.
3. Edit the PR description to use `Closes #NNN` instead, label is
removed but the bot comment stays.
## How to test (required)
1. Open a draft PR with no issue link → verify label and comment are
applied.
2. Edit the description to add `Refs #NNN` → verify label and comment
are both removed.
3. Repeat with `Closes #NNN`, `Fixes #NNN`, `References #NNN`.
## Risk / compatibility / migration
No Risk, CI workflow only.
## Linked issue
Closes#4556Closes#4557Closes#4144Closes#4151Closes#4152Closes#4153Closes#4229Closes#4356Closes#4401Closes#4403Closes#4519Closes#4523Closes#4390Closes#4414Closes#4484
## Summary / motivation
After 4 months of exploring Briefcase for packaging, we're confident it
solves most problems with the uv launcher with less complexity and at a
lower maintenance cost, especially with the parallel work on the release
CI, which we already used to make 5 releases.
This also removes platform-specific packaging/signing code used to
produce macOS dmg files and Windows NSIS installers, which is now
largely handled by Briefcase and the release CI.
The custom install scripts for Linux are mostly preserved under
qt/installer/linux-template and used in the Briefcase build.
## How to test
- [ ] No build problems.
- [ ] The `tools/build-installer` scripts still work.
- [ ] No CI/release workflow issues.
- [ ] No user-visible changes in dev environment and Briefcase build.
## Summary
- add a generated `docs-site/` Mintlify proof of concept for a unified
Anki docs site
- migrate the desktop manual, AnkiMobile docs, FAQs, add-on docs,
translation docs, release notes, legacy docs, and repo-local Sphinx/MyST
developer docs into the POC tree
- add a migration helper that preserves mdBook ordering, handles common
MDX incompatibilities, and regenerates the landing-page-inspired
Mintlify styling
- apply minimal styling based on the current Anki landing page: Anki
logo, Hanken Grotesk, blue primary color, subtle surfaces, and compact
nav treatment
## Validation
- `uv run --with ty ty check tools/mintlify_poc_migrate.py`
- `source ~/.nvm/nvm.sh && nvm use 22.15.0 && mint validate`
- previewed locally with `mint dev --port 3000` and checked the home
page/developer docs in browser
## Notes
This is intentionally a draft POC. It does not remove the existing
Sphinx or mdBook docs flows yet; it demonstrates what bringing the
sources into this repo and building from a single Mintlify root could
look like.
---------
Co-authored-by: Andrew Sanchez <andrewsanchez@users.noreply.github.com>
Co-authored-by: Luc Mcgrady <lucmcgrady@gmail.com>
Co-authored-by: Abdo <abdo@abdnh.net>
## Linked issue
Closes#4986Closes#4985
## Summary
- Move the complexipy-diff check to Ninja so it can be run locally as
part of `./ninja check`.
- Fix
[SARIF](https://docs.github.com/en/code-security/reference/code-scanning/sarif-files/sarif-support)
results not being uploaded if the Complexipy check fails.
## Steps to reproduce (before)
complexipy-diff was only run on CI and pre-push automatically. It was
not covered by ninja check, which is not consistent with most tests and
tools.
## How to test (after)
- Run `./ninja check:complexipy-diff` and confirm it passes.
- Introduce some complex Python change (e.g. add some nested if
statements) so that the check fails now.
- Update your pre-commit config: `./out/extracted/uv/uv run pre-commit
install`.
This triggers the check-linked-issue.yml workflow on the `labeled`
event, so that if the `hotfix` label is added later after the PR is
opened, the workflow is rerun.
## Summary / motivation
The `check-linked-issue` workflow was triggering with the `pull_request`
event. This caused the step "Apply missing-issue label and comment" to
fail with:
> HttpError: Resource not accessible by integration
Switching to `pull_request_target` makes the workflow run in the context
of the base repository instead of the fork, so the declared write
permissions are honoured.
The change is safe because this workflow never checks out PR code. It
only reads PR metadata (body, labels) through the API. There is no risk
of running untrusted code with elevated permissions.
## Steps to reproduce
1. Open a PR from an external fork that has no linked issue.
2. Observe the "Check Linked Issue" CI job failing with
`HttpError: Resource not accessible by integration`.
## Before / after behavior
**Before:** workflow crashes on every fork PR that lacks a linked issue.
**After:** workflow applies the label and posts the comment as intended.
## Linked issue
Closes#4815
## Summary
This adds [Complexipy](https://github.com/rohaquinlop/complexipy) for
detecting complex Python code:
- The `check:complexity` Ninja actions use a high threshold (50) for now
to avoid failing on existing complex code.
- `just complexipy-diff` is intended for linting new code in PR CI and
uses 15 as the threshold. See
https://rohaquinlop.github.io/complexipy/usage-guide/#ratchet-mode
## How to test
- Run `./ninja check:complexity` locally and confirm it passes.
- Test diff mode: `just complexipy-diff main`.
Fixes#4936
## Problem
The `check-linked-issue` workflow runs on every `opened` and `edited` PR
event. When a PR had no linked issue, each edit triggered a new bot
comment, resulting in duplicates (e.g. #4934).
## Solution
Before posting the comment, list the existing PR comments and skip
posting if the bot has already left one with the same message.
The `missing-issue` label re-application is harmless since GitHub
deduplicates labels automatically.
<!--
Title (for the Pull Request title field at the top):
Use a short prefix so the change type is obvious. You do not need to
repeat it in the body below.
Examples:
- fix: — bugfix
- feat: — feature
- refactor: — internal change without user-facing feature
- docs: — documentation only
- chore: — tooling, CI, deps, build housekeeping
- test: — tests only
-->
## Linked issue (required)
<!-- Fixes#123 / Closes#123 / Refs #123 -->
closes#4722
## Summary / motivation (required)
<!-- What this PR does and why. For larger changes, add enough context
for reviewers. -->
I used this nice script I found:
https://gist.github.com/onnimonni/3462f958c7d235417863651974514525
For the reasons behind this change see:
- #4722
## Steps to reproduce (required, use N/A if not applicable)
<!-- Steps to reproduce: how to trigger the bug in the broken state (the
"before").
- Mainly for bugfixes;
- For bugs: numbered steps before the fix. For non-bugs: write N/A.
- use N/A for features, refactors, docs, chore, etc.
-->
(N/A)
## How to test (required)
<!--- How to test: how you verified the change (checks, unit tests,
manual steps, edge cases — the "after" or general validation). --->
See it run in my repo here:
https://github.com/Luc-Mcgrady/anki/actions/runs/26718877866
### Checklist (minimum)
- [X] I ran `./ninja check` or an equivalent relevant check locally.
- [ ] I added or updated tests when the change is non-trivial or
behavior changed.
### Details
<!-- Commands, manual steps, edge cases, and what you observed -->
## Before / after behavior (optional)
<!-- For bugfixes: behavior before vs after. For other types: N/A or a
short note. -->
## Risk / compatibility / migration (optional)
<!-- Breaking changes, rollout notes, or N/A for small / low-risk PRs
-->
## UI evidence (required for visual changes; otherwise N/A)
<!-- Screenshot or short video -->
## Scope
- [X] This PR is focused on one change (no unrelated edits).
## Linked issue
Closes#4816
## Summary / motivation
Adds two workflows to enforce the rule that every PR must be linked to
an existing issue:
- **check-linked-issue**: triggers on PR open/edit, applies the
`missing-issue` label and notifies the author if no linked issue is
found. Removes the label if the author later links one.
- **auto-close-missing-issue**: runs daily and closes any PR that has
had the `missing-issue` label for more than 4 days.
Hotfixes (title contains `hotfix`) and Dependabot PRs are exempt.
## How to test
1. Open a PR without a linked issue, the `missing-issue` label should be
applied and a comment posted.
2. Edit the PR description to add `Closes #<number>`, the label should
be removed.
3. Trigger the auto-close workflow manually via Actions → `Auto-close
PRs without linked issue` → Run workflow, and verify it closes PRs that
have had the label for over 4 days.
## Linked issue
Closes#4874
## Summary / motivation
Adds `tools/coverage/check-coverage-regression.py` to compare line
coverage percentages from the current PR against the baseline saved from
main (introduced in #4875). If any stack regresses beyond the
configured tolerance (0.10%), the CI fails with a clear message showing
the delta.
Stacks checked: Rust, python-pylib, python-qt, TypeScript.
## How to test
Try to add some new code without any tests. The Ci must fail.
## Before / after behavior
Before: no signal when a PR reduces coverage below the current main
level.
After: CI fails on `Check coverage regression` with output like:
```
[rust] REGRESSION: 62.64% -> 61.00% (delta: -1.64%, tolerance: 0.10%)
1 stack(s) with coverage regression: rust
```
## Linked issue
Closes#4873
## Summary
Build and package the fcitx5-qt6 plugin.
Latest release CI run:
https://github.com/ankitects/anki/actions/runs/26294296416
## How to test
1. Run installer build on Linux: `./ninja installer:build`.
2. Go to the Qt build directory
(`out/installer/build/anki/linux/zip/anki/app_packages/PyQt6/Qt6`) and
confirm you see the following files:
1.
`plugins/platforminputcontexts/libfcitx5platforminputcontextplugin.so`
2. `plugins/dbusaddons/libFcitx5Qt6DBusAddons.so*`
## Linked issue
Refs #4874
## Summary / motivation
Stores the coverage results from every push to `main` in a GitHub
Actions
cache (`coverage-baseline-linux-{sha}`). This is the foundation for a
follow-up PR that will compare PR coverage against this baseline and
fail
if any stack regresses.
No behavior change for PRs yet — the baseline is only saved, not used.
## Before / after behavior
Before: no coverage data persisted between CI runs.
After: each push to `main` saves `out/coverage/` as a cache entry, keyed
by commit SHA, retrievable by prefix `coverage-baseline-linux-`.
## Linked issue
Closes#4863
## Summary / motivation
Adds Playwright as the e2e test framework so contributors can write
browser-based tests against a real headless Anki instance. There was no
automated way to exercise mediasrv pages, SvelteKit routes, or the
`/_anki/` RPC surface from a browser, this PR establishes that harness.
Key pieces:
- `qt/tests/launch_anki_for_e2e.py` — spawns a throwaway Anki instance
(temp `ANKI_BASE`, `QT_QPA_PLATFORM=offscreen`). Pre-seeds `prefs21.db`
so Anki skips the language picker and profile chooser and goes straight
to serving mediasrv.
- `playwright.config.ts` — points `webServer` at the launcher; polls
`/favicon.ico` as the readiness probe.
- `ts/tests/e2e/` — `fixtures.ts` base and a sanity spec that verifies
mediasrv is reachable and a SvelteKit page hydrates.
- `justfile` — `just test-e2e` recipe; Chromium installed to
`out/playwright-browsers/`.
- CI — e2e step in `check-linux`; failed-run artifacts uploaded for 7
days.
- `docs/e2e-testing.md` — contributor guide covering setup, managed vs
reuse-server modes, and writing new tests.
## How to test
Build the project once, then run the e2e suite in managed mode (no
separate `./run` needed — the launcher is started automatically):
```shell
just build
just test-e2e
```
## Before / after behavior (optional)
Before: no browser-level test harness existed.
After: `just test-e2e` drives a real headless Anki instance via
Playwright.
## Risk / compatibility / migration
No production code changed. New dev-only files and CI step only.
Chromium is installed to `out/playwright-browsers/` (gitignored) and
does not affect the regular build.
---------
Co-authored-by: Abdo <abdo@abdnh.net>
## Linked issue
Closes#4859
## Summary
Add tests for the build_installer.py script with 100% coverage.
## How to test
Run `just test-py --coverage --html` and browse coverage data.
## Linked issue
Closes#4839
## Summary / motivation
Adds `cargo-llvm-cov`-based test coverage for the full Rust workspace.
Introduces `just test-rust --coverage` and `just test-rust --coverage
--html`, and wires Rust into the `just test --coverage` umbrella.
`cargo-llvm-cov` is installed on demand into `out/bin/` to avoid
polluting the global cargo install. The `llvm-tools-preview` rustup
component is now installed in CI so the tool can instrument binaries.
## How to test (required)
```sh
# Existing behavior unchanged
just test-rust
# Terminal summary
just test-rust --coverage
# Terminal summary + HTML report under out/coverage/rust/html/
just test-rust --coverage --html
# Umbrella (Rust + Python)
just test --coverage
just test --coverage --html
```
Note: first run of `--coverage` will install `cargo-llvm-cov` into
`out/bin/` (~30s). Subsequent runs skip the install step.
### Checklist
- [x] I ran `./ninja check` or an equivalent relevant check locally.
### Details
- `cargo-llvm-cov` pinned at `0.8.4`, installed into `out/bin/` via
`cargo install --root out`.
- `--workspace --locked` measures all crates and respects the lockfile.
- `llvm-tools-preview` added to `setup-anki` action so CI can instrument
Rust binaries.
- Coverage runs are slower than plain `just test-rust` because
`cargo-llvm-cov` rebuilds with instrumentation — this is expected.
## Before / after behavior
Before: no `just test-rust`, no Rust coverage support.
After: `just test-rust` runs Rust tests via ninja; `just test-rust
--coverage` runs them with `cargo-llvm-cov`
---------
Co-authored-by: Abdo <abdo@abdnh.net>
## Linked issue
Closes#4838
## Summary/motivation
Adds `coverage.py`-based test coverage for both Python test suites
(`pylib` and `qt`). Introduces `just test-py --coverage` and `just
test-py --coverage --html`, plus the `just test --coverage`.
Coverage reports are written to `out/coverage/`.
## How to test
```sh
# Existing behavior unchanged
just test-py
# Terminal summary + enforces thresholds
just test-py --coverage
# Terminal summary + HTML reports under out/coverage/
just test-py --coverage --html
# Umbrella (Python only for now)
just test --coverage
just test --coverage --html
```
### Checklist (minimum)
- [x] I ran `./ninja check` or an equivalent relevant check locally.
### Details
- `coverage` dependency pinned to >=7.13.5 in pyproject.toml.
- The `coverage` umbrella recipe currently delegates to Python only for
now
## Before / after behavior
Before: no `just test-py`, no coverage support.
After: `just test-py` runs Python tests via ninja; `just test-py
--coverage`
runs them with `coverage.py` and enforces minimum line coverage.
---------
Co-authored-by: Abdo <abdo@abdnh.net>
Pass the `--notes-start-tag` argument to `gh release` with the latest
release tag. Without this, the 26.05b1 release was including notes for
the 25.09.2 release for some reason.
## Linked issue
#4678
## Summary
This enables native Windows ARM64 builds for Briefcase. Depends on #4797
## How to test
- Run `./tools/ninja installer` in a Windows ARM64 machine.
- Check the architecture of the installer under `./out/installer/dist`
by going to Properties > Compatibility and confirming emulation settings
are disabled.
- Install the package and confirm Anki.exe is a native binary.
- Open Anki, go to the [debug
console](https://docs.ankiweb.net/misc.html#debug-console) and run the
following code to check the architecture of the Python build:
```python
import platform
print(platform.machine(), platform.python_compiler())
```
Closes#4793
- Add `workflow_dispatch` trigger to CI (with macOS/Windows support)
- Allow prepare-release and release workflows from any branch
- Add `skip-ci-check` input for hotfix releases
- Add `just release::prepare` and `just ci` recipes
- Make `qt/release/build.sh` find uv in CI and local builds
- Change publish-testpypi environment from testpypi to release
- Add anki-release wheel build step
---------
Co-authored-by: Andrew Sanchez <andrewsanchez@users.noreply.github.com>
Co-authored-by: Fernando Lins <1887601+fernandolins@users.noreply.github.com>
migrates Anki Desktop packaging from the legacy
NSIS/uv-based installer to [BeeWare
Briefcase](https://briefcase.readthedocs.io/). This branch integrates
work from many related issues and PRs to deliver cross-platform native
installers (MSI on Windows, .app on macOS, PyInstaller on Linux) with
code signing, notarization, and file association support.
## Integrated PRs
- #4585 — Set up Briefcase
- #4596 — Add Briefcase icons
- #4598 — Handle Briefcase file associations
- #4601 — Add Briefcase app permissions
- #4609 — Customize Briefcase's MSI installer
- #4616 — Set up Briefcase code signing and notarization
- #4618 — Fix Briefcase packaging for x86 Macs
- #4623 — Customize Briefcase's Linux template
- #4627 — List required Debian packages for Briefcase installer
- #4630 — Update Briefcase's Windows template
- #4631 — Rewrite Linux install/uninstall scripts for PyInstaller
- #4638 — Use PyInstaller on Linux
- #4645 — Update installer docs
- #4654 — Disable Briefcase's universal builds for macOS
- #4672 — Deal with existing NSIS installations in MSI installer
- #4676 — Remove duplicate Briefcase icons
- #4677 — Tweak Linux scripts for new installer
- #4709 — Add anki-console.bat to Briefcase's Windows package
## Related Issues
- #4557 — Evaluate BeeWare Briefcase for Anki packaging and distribution
- #4678 — Support native Windows ARM64 builds for Briefcase
- #4688 — Linux installer: migrate to PyInstaller and rewrite install
scripts
- #4689 — Investigate startup performance with Briefcase
- #4690 — Specify required Linux system packages for Briefcase
- #4691 — Investigate Windows ARM64 support with Briefcase
- #4692 — Test on Linux ARM with Briefcase
- #4693 — Separate ARM and Intel macOS releases
- #4694 — Update developer documentation for Briefcase installer
- #4695 — Support upgrade/downgrade with the Briefcase installer
- #4696 — Update user documentation for new installer
- #4702 — Update Briefcase's Windows template with upstream security fix
and OS version check
- #4703 — Follow-up tweaks to Linux install/uninstall scripts
## Related PRs
- #4619 — Enable Windows ARM64 support
- #4632 — Release action
---------
Co-authored-by: Abdo <abdo@abdnh.net>
Co-authored-by: Andrew Sanchez <andrewsanchez@users.noreply.github.com>
Co-authored-by: Fernando Lins <1887601+fernandolins@users.noreply.github.com>
## What
- Set `open-pull-requests-limit` to **3** for each `package-ecosystem`
in [`.github/dependabot.yml`](.github/dependabot.yml) so we allow a few
more concurrent Dependabot version-update PRs per stack while still
bounding review load.
## Why
- A slightly higher cap lets more ecosystems progress in parallel when
the queue is healthy, without going back to the default (5) or an
unbounded backlog of open PRs.
## Linked issue (required)
Partially related to #4722
## Summary/motivation (required)
Weekly (by default), Dependabot checks for updates and opens a PR for
each dependency update. This generates numerous PRs that require human
review time and GitHub Actions overhead.
By adding the Dependabot configuration file, we changed the frequency to
monthly and grouped the PRs by ecosystem.
## Scope
This PR does:
- [x] Fewer open dependency PRs (minor/patch grouped per ecosystem)
- [x] Less CI / GitHub Actions churn (monthly cadence + fewer PRs)
- [x] Clearer review queue (updates batched by Rust / JS / Python /
Actions)
This PR does not:
❌ Group major version bumps (still separate PRs)
❌ Shorten individual CI runs
This PR introduces a standardized pull request template setup to improve
review quality and consistency.
## Summary
- Adds a new default PR template at `.github/pull_request_template.md`.
- Standardizes PR submissions with consistent sections for context and
validation.
- Establishes a single source of truth for PR guidance.
## Why
- A unified template improves review quality and consistency.
- It reduces ambiguity for contributors when describing changes and test
coverage.
- Centralizing guidance makes future maintenance simpler.
## Changes
Revert changes to the CONTRIBUTORS file check introduced in
https://github.com/ankitects/anki/pull/4593#discussion_r2908422240
The main problem with checking the actual emails in the file instead of
the git log is that the check will inevitably fail when the PR author
occasionally makes a commit using the GitHub UI. The solution for this
used to be to also make a change to the file using the GitHub UI. This
stopped working after the recent change, except if the author lists
multiple emails.
(for upgrading users, please see the notes at the bottom)
Bazel brought a lot of nice things to the table, such as rebuilds based on
content changes instead of modification times, caching of build products,
detection of incorrect build rules via a sandbox, and so on. Rewriting the build
in Bazel was also an opportunity to improve on the Makefile-based build we had
prior, which was pretty poor: most dependencies were external or not pinned, and
the build graph was poorly defined and mostly serialized. It was not uncommon
for fresh checkouts to fail due to floating dependencies, or for things to break
when trying to switch to an older commit.
For day-to-day development, I think Bazel served us reasonably well - we could
generally switch between branches while being confident that builds would be
correct and reasonably fast, and not require full rebuilds (except on Windows,
where the lack of a sandbox and the TS rules would cause build breakages when TS
files were renamed/removed).
Bazel achieves that reliability by defining rules for each programming language
that define how source files should be turned into outputs. For the rules to
work with Bazel's sandboxing approach, they often have to reimplement or
partially bypass the standard tools that each programming language provides. The
Rust rules call Rust's compiler directly for example, instead of using Cargo,
and the Python rules extract each PyPi package into a separate folder that gets
added to sys.path.
These separate language rules allow proper declaration of inputs and outputs,
and offer some advantages such as caching of build products and fine-grained
dependency installation. But they also bring some downsides:
- The rules don't always support use-cases/platforms that the standard language
tools do, meaning they need to be patched to be used. I've had to contribute a
number of patches to the Rust, Python and JS rules to unblock various issues.
- The dependencies we use with each language sometimes make assumptions that do
not hold in Bazel, meaning they either need to be pinned or patched, or the
language rules need to be adjusted to accommodate them.
I was hopeful that after the initial setup work, things would be relatively
smooth-sailing. Unfortunately, that has not proved to be the case. Things
frequently broke when dependencies or the language rules were updated, and I
began to get frustrated at the amount of Anki development time I was instead
spending on build system upkeep. It's now about 2 years since switching to
Bazel, and I think it's time to cut losses, and switch to something else that's
a better fit.
The new build system is based on a small build tool called Ninja, and some
custom Rust code in build/. This means that to build Anki, Bazel is no longer
required, but Ninja and Rust need to be installed on your system. Python and
Node toolchains are automatically downloaded like in Bazel.
This new build system should result in faster builds in some cases:
- Because we're using cargo to build now, Rust builds are able to take advantage
of pipelining and incremental debug builds, which we didn't have with Bazel.
It's also easier to override the default linker on Linux/macOS, which can
further improve speeds.
- External Rust crates are now built with opt=1, which improves performance
of debug builds.
- Esbuild is now used to transpile TypeScript, instead of invoking the TypeScript
compiler. This results in faster builds, by deferring typechecking to test/check
time, and by allowing more work to happen in parallel.
As an example of the differences, when testing with the mold linker on Linux,
adding a new message to tags.proto (which triggers a recompile of the bulk of
the Rust and TypeScript code) results in a compile that goes from about 22s on
Bazel to about 7s in the new system. With the standard linker, it's about 9s.
Some other changes of note:
- Our Rust workspace now uses cargo-hakari to ensure all packages agree on
available features, preventing unnecessary rebuilds.
- pylib/anki is now a PEP420 implicit namespace, avoiding the need to merge
source files and generated files into a single folder for running. By telling
VSCode about the extra search path, code completion now works with generated
files without needing to symlink them into the source folder.
- qt/aqt can't use PEP420 as it's difficult to get rid of aqt/__init__.py.
Instead, the generated files are now placed in a separate _aqt package that's
added to the path.
- ts/lib is now exposed as @tslib, so the source code and generated code can be
provided under the same namespace without a merging step.
- MyPy and PyLint are now invoked once for the entire codebase.
- dprint will be used to format TypeScript/json files in the future instead of
the slower prettier (currently turned off to avoid causing conflicts). It can
automatically defer to prettier when formatting Svelte files.
- svelte-check is now used for typechecking our Svelte code, which revealed a
few typing issues that went undetected with the old system.
- The Jest unit tests now work on Windows as well.
If you're upgrading from Bazel, updated usage instructions are in docs/development.md and docs/build.md. A summary of the changes:
- please remove node_modules and .bazel
- install rustup (https://rustup.rs/)
- install rsync if not already installed (on windows, use pacman - see docs/windows.md)
- install Ninja (unzip from https://github.com/ninja-build/ninja/releases/tag/v1.11.1 and
place on your path, or from your distro/homebrew if it's 1.10+)
- update .vscode/settings.json from .vscode.dist
