Commit Graph

102 Commits

Author SHA1 Message Date
Matt Godbolt
ddb0266645 Convert propscheck.py to TypeScript/Vitest tests (#8404)
## Summary

Replaces the Python `propscheck.py` validation script with TypeScript
code that:
- Uses a testable validator module (`lib/properties-validator.ts`)
- Has comprehensive unit tests (71 tests in
`test/properties-validation-tests.ts`)
- Integrates with the existing property loading infrastructure
- Runs as part of the normal test suite

## Changes

- **New validator module** (`lib/properties-validator.ts`):
- Raw file validation: duplicate keys, empty list elements, typos,
suspicious paths
  - Orphan detection: compilers, groups, formatters, tools, libraries
  - Cross-file duplicate compiler ID detection
  - Disabled allowlist support (`# Disabled: id1 id2`)
- Invalid property format detection (leverages `parseProperties()` with
new `collectErrors` option)
  - Missing compilers list detection for language files

- **Property library enhancement** (`lib/properties.ts`):
- Added `collectErrors` option to `parseProperties()` for structured
error collection

- **New npm script**: `npm run test:props` for quick property validation
  - Set `CHECK_LOCAL_PROPS=true` to include `.local.properties` files

- **Updated references**:
  - CI workflow no longer calls propscheck.py
  - Pre-commit hook uses `npm run test:props`
  - CE Properties Wizard uses the new validation

- **Removed**: `etc/scripts/util/propscheck.py` and `propschecktest.py`

## Test plan

- [x] All 71 unit tests pass
- [x] Integration tests validate real config files
- [x] `npm run test:props` works standalone
- [x] CE Properties Wizard validation works
- [x] CI passes

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 19:03:00 -05:00
Matt Godbolt
335f556297 Fix Sentry CLI v3.0.0 breaking change (#8336)
## Summary
- Sentry CLI 3.0.0 removed the `releases files` subcommand
- Migrate to the new `sourcemaps upload --release` command syntax
- Fixes deploy failures:
https://github.com/compiler-explorer/compiler-explorer/actions/runs/20253797661/job/58151514156

See: https://github.com/getsentry/sentry-cli/releases/tag/3.0.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 20:26:03 -06:00
Matt Godbolt
eca351fa49 Add automated duplicate issue detection (#8171)
## Summary
Implements automated detection of potential duplicate issues using
GitHub Actions.

## Implementation Details
- Uses `wow-actions/potential-duplicates@v1` action
- Triggers on issue `opened` and `edited` events
- Configured with **90% similarity threshold** to minimize false
positives
- Automatically adds `potential-duplicate` label and comment with links
to similar issues

## Rationale for 90% Threshold
Based on manual testing with the `gh_tool` CLI:
- 85% threshold produced too many false positives from template-based
matches
- True duplicates typically have 90%+ similarity
- Uses Damerau-Levenshtein distance algorithm (same as manual tool)

## Testing
-  YAML syntax validated
-  Pre-commit hooks passed (lint + ts-check)
-  Created `potential-duplicate` label in repository
- Manual testing of similar configuration showed good results at 90%
threshold

## Dependencies
- Depends on #8166 (add-gh-tools-cli branch) being merged first
- This PR adds the automation layer on top of the manual tooling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-07 15:11:16 -05:00
Matt Godbolt
4cb1416c2a Add gh_tool CLI for GitHub repository automation (#8170)
This PR adds a new Python CLI tool for automating GitHub repository
management tasks.

## Overview

The initial implementation provides duplicate issue detection using text
similarity analysis. This is the first step toward automating repository
triage tasks.

## Features

- **Click-based CLI** with subcommands for future extensibility
- **find-duplicates command** for detecting duplicate issues using text
similarity
- Uses **gh CLI** for GitHub API access (no token management needed)
- Text similarity using `difflib.SequenceMatcher` (ratio-based
algorithm)
- Configurable similarity threshold (default: 0.6)
- Progress bar for long-running comparisons
- Age filtering support (`--min-age` parameter)
- Standard Python src-layout with **uv** for dependency management
- **Comprehensive test suite** with pytest (integrated into CI)

## Project Structure

```
etc/scripts/gh_tool/
├── src/gh_tool/          # Main package
│   ├── cli.py            # Click-based CLI interface
│   └── duplicate_finder.py  # Core duplicate detection logic
├── tests/                # Test suite
│   └── test_duplicate_finder.py
├── docs/                 # Documentation
│   ├── TRIAGE-CRITERIA.md    # Triage guidelines from manual review
│   └── PHASE1-FINDINGS.md    # Historical analysis of 855 issues
├── pyproject.toml        # Package configuration
└── README.md             # Usage documentation
```

## Usage

```bash
cd etc/scripts/gh_tool
uv sync
uv run gh_tool find-duplicates /tmp/report.md
```

**Options:**
- `--threshold FLOAT` - Similarity threshold 0-1 (default: 0.6)
- `--state {all,open,closed}` - Issue state to check (default: open)
- `--min-age DAYS` - Only check issues older than N days (default: 0)
- `--limit INTEGER` - Maximum number of issues to fetch (default: 1000)
- `--repo TEXT` - GitHub repository in owner/repo format (default:
compiler-explorer/compiler-explorer)

**Example:**
```bash
# Find high-confidence duplicates in open issues
uv run gh_tool find-duplicates /tmp/report.md --threshold 0.85

# Check all issues older than 30 days
uv run gh_tool find-duplicates /tmp/report.md --state all --min-age 30
```

## Testing

The tool includes comprehensive test coverage:
- Unit tests for similarity calculation
- Integration tests for duplicate detection
- Edge case handling (transitive grouping, age filtering, threshold
sensitivity)
- Report generation validation

**Run tests:**
```bash
cd etc/scripts/gh_tool
uv run pytest -v
```

Tests are integrated into CI and run on every push.

## Documentation

- **`README.md`**: Complete usage guide with examples
- **`docs/TRIAGE-CRITERIA.md`**: Comprehensive triage guidelines
developed during manual review of 22+ issues
- **`docs/PHASE1-FINDINGS.md`**: Historical analysis context from
initial 855 issue review

## CI Integration

The tool is integrated into the GitHub Actions workflow:
- `uv` is installed via `astral-sh/setup-uv@v6`
- Tests run automatically on every push
- Ensures tool remains functional as codebase evolves

## Next Steps

Future enhancements planned for follow-up PRs:
- GitHub Action for automatic duplicate detection on new issues
- Additional automation tools (upstream health checker, label validator,
etc.)
- Automated triage reports

## Changes in this PR

-  Core duplicate detection implementation
-  Comprehensive test suite (192 lines)
-  CI integration
-  Complete documentation
-  Example triage criteria and findings

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-07 14:50:22 -05:00
Patrick Quist
d146b15ea3 Update deploy-win.yml 2025-06-09 20:24:26 +02:00
Patrick Quist
1f8984b020 Update test-win.yml 2025-06-09 20:24:08 +02:00
Matt Godbolt
54c942ba76 Replace nopt with commander.js for argument parsing (#7673)
- Replace nopt with commander.js for better command-line argument parsing
- Add automatic help generation with detailed descriptions
- Maintain backward compatibility with existing arguments
- Remove unused nopt dependency from package.json

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-05-12 12:49:28 -05:00
Matt Godbolt
f493348115 Maybe resurrect coverage? 2025-04-19 23:01:06 -05:00
Patrick Quist
68d53ee163 Update test-and-deploy.yml 2025-04-08 18:31:19 +02:00
Rupert Tombs
5eeded45c4 Test the numba wrapper in test-and-deploy (#7413)
- Resolve comments from #5592:
- Replace a silly indirect import
https://github.com/compiler-explorer/compiler-explorer/pull/5592#discussion_r1962004963
- Run `test_numba_wrapper` in `.github/workflows/test-and-deploy.yml`.
https://github.com/compiler-explorer/compiler-explorer/pull/5592#discussion_r1962004131
- Patch minor errors in `etc/scripts/util/propschecktest.py` and also
test it.
2025-02-24 09:58:31 -06:00
Matt Godbolt
3aeed42066 Upgrade to node 20 minimum, target node 22 (#7343)
Co-authored-by: Mats Jun Larsen <mats@jun.codes>
2025-02-18 12:43:21 -06:00
Mats Jun Larsen
dc5f3cf43b Deduplicate npm scripts used for CI (#7356)
There's nothing special with these anymore. They are the same both
locally and in CI now, because our Vitest and Biome setups don't behave
differently there
2025-02-04 03:28:21 +09:00
Mats Jun Larsen
64397a3895 Use latest version of nodejs 18 in minimum build (#7206) 2024-12-09 20:36:24 +09:00
Partouf
15dcf0d42a add extra check 2024-09-25 14:33:26 +02:00
Matt Godbolt
36cacb20ba Put the build version into the run-name 2024-06-19 10:58:51 -05:00
Matt Godbolt
db9e3cbefe Bump to v6 cypress run config (#6484)
Seems like a decent thing to do. cypress should already be using node 20
though
2024-05-13 23:10:56 -05:00
Patrick Quist
2f6ca2323f Update test-frontend.yml 2024-05-14 05:46:08 +02:00
Partouf
4c7d7a4a20 change up windows workflow, regular shell (pwsh) continues on errors 2024-04-13 21:11:41 +02:00
Partouf
a5d1e44264 stick windows to node v20.12.1, 20.12.2 seems to be broken 2024-04-13 19:13:13 +02:00
Patrick Quist
03ebcdd2b4 add workflow minimum support build (#6175) 2024-02-21 22:07:01 +01:00
renovate[bot]
ee47a9be8f chore(deps): update peter-evans/create-pull-request action to v6 (#6168)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[peter-evans/create-pull-request](https://togithub.com/peter-evans/create-pull-request)
| action | major | `v5` -> `v6` |

---

### Release Notes

<details>
<summary>peter-evans/create-pull-request
(peter-evans/create-pull-request)</summary>

###
[`v6`](https://togithub.com/peter-evans/create-pull-request/compare/v5...v6)

[Compare
Source](https://togithub.com/peter-evans/create-pull-request/compare/v5...v6)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 8pm on monday" in timezone
America/Chicago, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/compiler-explorer/compiler-explorer).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMDAuMCIsInVwZGF0ZWRJblZlciI6IjM3LjIwMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-19 18:18:20 -06:00
renovate[bot]
e2adcabd65 chore(deps): update github/codeql-action action to v3 (#6166)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github/codeql-action](https://togithub.com/github/codeql-action) |
action | major | `v2` -> `v3` |

---

### Release Notes

<details>
<summary>github/codeql-action (github/codeql-action)</summary>

### [`v3`](https://togithub.com/github/codeql-action/compare/v2...v3)

[Compare
Source](https://togithub.com/github/codeql-action/compare/v2...v3)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 8pm on monday" in timezone
America/Chicago, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/compiler-explorer/compiler-explorer).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMDAuMCIsInVwZGF0ZWRJblZlciI6IjM3LjIwMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-19 18:00:21 -06:00
Matt Godbolt
a1d6f9422d Fix coverage (#6157) 2024-02-18 15:41:12 -06:00
Marc Auberer
f69c538415 [CI] Fix labeler configuration vor labler@v5 (#6115)
This pr fixes the failing labeler action labeling pull requests, which
was probably broken by renovate when upgrading the action from v4 to v5.
I wasn't able to test the configuration, but let's hope it works.
2024-02-10 19:26:30 +01:00
Patrick Quist
5fa3e359a4 attempt to fix error 2024-02-09 18:21:25 +01:00
Matt Godbolt
56d0159947 Combines two renovates into one! (#6101)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-05 21:27:02 -06:00
renovate[bot]
b030f0a809 chore(deps): update actions/labeler action to v5 (#6093)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/labeler](https://togithub.com/actions/labeler) | action |
major | `v4` -> `v5` |

---

### Release Notes

<details>
<summary>actions/labeler (actions/labeler)</summary>

### [`v5`](https://togithub.com/actions/labeler/compare/v4...v5)

[Compare Source](https://togithub.com/actions/labeler/compare/v4...v5)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 8pm on monday" in timezone
America/Chicago, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/compiler-explorer/compiler-explorer).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNzMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjE3My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-05 20:36:17 -06:00
renovate[bot]
09e9ba167f chore(deps): update actions/setup-node action to v4 (#6094)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/setup-node](https://togithub.com/actions/setup-node) | action
| major | `v3` -> `v4` |

---

### Release Notes

<details>
<summary>actions/setup-node (actions/setup-node)</summary>

### [`v4`](https://togithub.com/actions/setup-node/compare/v3...v4)

[Compare
Source](https://togithub.com/actions/setup-node/compare/v3...v4)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 8pm on monday" in timezone
America/Chicago, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/compiler-explorer/compiler-explorer).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNzMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjE3My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-05 20:36:10 -06:00
renovate[bot]
2ee0db36e3 chore(deps): update actions/github-script action to v7 (#6092)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/github-script](https://togithub.com/actions/github-script) |
action | major | `v6` -> `v7` |

---

### Release Notes

<details>
<summary>actions/github-script (actions/github-script)</summary>

### [`v7`](https://togithub.com/actions/github-script/compare/v6...v7)

[Compare
Source](https://togithub.com/actions/github-script/compare/v6...v7)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 8pm on monday" in timezone
America/Chicago, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/compiler-explorer/compiler-explorer).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNzMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjE3My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-05 20:33:39 -06:00
Matt Godbolt
7e139412a3 Move runners to 22.04. Partially subsumes #4950 (#5878)
See #4950
2023-12-11 20:53:32 -06:00
Matt Godbolt
e8a1f3fab2 Bump to not-about-to-be-removed version of github-script 2023-09-27 23:18:32 -05:00
Matt Godbolt
5d776aaae3 Replace ts-node-esm with direct calls to node to support node 20 (#5521)
- works both v18 and v20
- see https://github.com/TypeStrong/ts-node/issues/1997
2023-09-27 21:57:18 -05:00
Marc Auberer
fb26b2d9a1 Update checkout action to v4 (#5479)
Upgrade checkout action to v4 to make use of Node 20.
Node 16, which v3 is using, reaches end of life on 2023-09-11.
2023-09-16 17:13:45 -05:00
Matt Godbolt
c9294fccbb Attempt at using a tag script instead of a (soon to be deprecated) workflow (#5358)
Fix for...

> The following actions uses node12 which is deprecated and will be
forced to run on node16: tvdias/github-tagger@v0.0.2. For more info:
https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
2023-08-07 22:43:59 -05:00
Goooler
36213d12de Renovate updates (#5291)
We can kick these things by @renovate-bot.

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-07-30 18:17:20 -05:00
Mohammed Keyvanzadeh
abf92f10f6 tools: update GitHub Actions workflows (#4943)
- Update the actions in the GitHub Actions workflows to their latest versions.
- Run the workflows on the latest versions of the platforms where appropriate.
- Set the `check-latest` field of the `actions/setup-node` action to `true` to use the latest release of the specified Node.js version instead of using the cached one when there's a new release available.
- Explicitly specify the action versions instead of using the `latest` tag.
2023-04-05 11:24:31 -05:00
Matt Godbolt
54a99f6d51 Upgrade all minor versions; format; bump sonar and css minimizer (#4920) 2023-04-01 20:29:06 -05:00
Matt Godbolt
a8f832f80c Bump to latest cypress (#4853)
Latest cypress:
- updated all the tests
- fixed up warnings
- got rid of isolation stuff, it's now isolated by default
- added simple docs
- added `--noLocal` to prevent any local config from screwing up when
testing locally
- store screenshot artifacts if things go wrong
2023-03-14 10:00:40 -05:00
Matt Godbolt
fb4b7bb16e Bump used node version to 18 (#4778)
Still assume node 16 compatibility for now.
2023-02-25 12:43:00 -06:00
Jeremy Rifkin
ffe1cf7ce6 Fix cypress timeout issue (#4776)
This should fix the issue of cypress CI frequently timing out and having
to manually re-run to get checks to be OK
2023-02-25 10:13:23 -06:00
Patrick Quist
396182e31f Windows build and deploy script (#4547) 2023-01-10 22:14:09 +01:00
Rubén Rincón Blanco
a615ea58e9 Test only in Chrome for now 2022-11-09 16:14:57 +01:00
Rubén Rincón Blanco
d41eb5c269 Run Cypress in both firefox & chrome (#4106)
* Run Cypress in both firefox & chrome

* Use matrix for frontend testing workflow

Done following https://github.com/browser-actions/examples/blob/master/.github/workflows/build.yml
2022-11-09 03:24:40 +01:00
Matt Godbolt
a4449cb012 Workflow bump v2 v3 checkout (#4140)
* Workflow bump v2 v3 checkout
* Bump labeler
2022-10-11 17:27:26 -05:00
Rubén Rincón Blanco
52af6ecbe7 Ask git to use LF line endings in windows CI (#4039) 2022-09-08 23:32:42 +02:00
Rubén Rincón Blanco
8d6f171197 Cypress test to open all panes (#3953) 2022-09-04 17:31:36 +02:00
Rubén Rincón Blanco
16b176235a JBOF: Just a Bunch of Fixes (#3957)
- Renames orphancompiler.py script to propscheck.py, the name fits better
- Fixes Go props to make them easier to maintain
- Alphabetically sorts some listings in the code
- Adds tests to the props checker script (No idea how to integrate those so it's manual for now)
- Adds 2 missing limit cases for Motd filtering
2022-08-13 01:39:41 +02:00
Mats Larsen
18160be477 Use eslint-plugin-prettier to fix and check formatting problems (#3847) 2022-07-12 22:56:03 +02:00
Mats Larsen
e24b2f72be Remove unused prettier format script (#3646) 2022-05-15 20:36:13 +01:00
Matt Godbolt
31104f3230 Use our own fork of approvals to fix a bunch of security warnings. (#3651)
* Use our own fork of approvals to fix a bunch of security warnings.
* use node 16

Co-authored-by: partouf <partouf@gmail.com>
2022-05-13 07:44:29 -05:00