CE Properties Wizard: Interactive tool for adding compilers (#7934)

This commit is contained in:
Patrick Quist
2025-10-04 12:11:58 +02:00
committed by GitHub
parent e39d23b96b
commit 16af4186ad
17 changed files with 5653 additions and 9 deletions

2
.gitignore vendored
View File

@@ -37,3 +37,5 @@ newrelic_agent.log
# Claude local settings # Claude local settings
.claude/settings.local.json .claude/settings.local.json
.aider* .aider*
etc/scripts/ce-properties-wizard/ce_properties_wizard/__pycache__

View File

@@ -110,6 +110,11 @@ If you want to point it at your own GCC or similar binaries, either edit the `et
else make a new one with the name `LANG.local.properties`, substituting `LANG` as needed. `*.local.properties` files else make a new one with the name `LANG.local.properties`, substituting `LANG` as needed. `*.local.properties` files
have the highest priority when loading properties. have the highest priority when loading properties.
For a quick and easy way to add local compilers, use the
[CE Properties Wizard](etc/scripts/ce-properties-wizard/) which automatically detects and configures compilers
for [30+ languages](etc/scripts/ce-properties-wizard/README.md#supported-languages).
See [Adding a Compiler](docs/AddingACompiler.md) for more details.
If you want to support multiple compilers and languages like [godbolt.org](https://godbolt.org), you can use the If you want to support multiple compilers and languages like [godbolt.org](https://godbolt.org), you can use the
`bin/ce_install install compilers` command in the [infra](https://github.com/compiler-explorer/infra) project to install `bin/ce_install install compilers` command in the [infra](https://github.com/compiler-explorer/infra) project to install
all or some of the compilers. Compilers installed in this way can be loaded through the configuration in all or some of the compilers. Compilers installed in this way can be loaded through the configuration in

View File

@@ -3,7 +3,54 @@
This document explains how to add a new compiler to Compiler Explorer ("CE" from here on), first for a local instance, This document explains how to add a new compiler to Compiler Explorer ("CE" from here on), first for a local instance,
and then how to submit PRs to get it into the main CE site. and then how to submit PRs to get it into the main CE site.
## Configuration ## Quick method: Using ce-properties-wizard
The easiest way to add a compiler to your local Compiler Explorer instance is to use the `ce-properties-wizard` tool. This interactive command-line tool automatically detects compiler information and updates your configuration files.
### Basic usage
From the Compiler Explorer root directory:
```bash
# Interactive mode - guides you through the process
etc/scripts/ce-properties-wizard/run.sh
# Path-first mode - provide compiler path directly
etc/scripts/ce-properties-wizard/run.sh /usr/bin/g++-13
# Fully automated mode - accepts all defaults
etc/scripts/ce-properties-wizard/run.sh /usr/bin/g++-13 --yes
```
### Examples
Add a custom GCC installation:
```bash
etc/scripts/ce-properties-wizard/run.sh /opt/gcc-14.2.0/bin/g++
```
Add a cross-compiler:
```bash
etc/scripts/ce-properties-wizard/run.sh /usr/bin/arm-linux-gnueabihf-g++ \
--name "ARM GCC 11.2" \
--group arm-gcc \
--yes
```
The wizard will:
- Automatically detect the compiler type, version, and language
- Generate appropriate compiler IDs and display names
- Add the compiler to the correct properties file
- Suggest appropriate groups for organization
- Validate the configuration with `propscheck.py`
For more options and examples, see the [ce-properties-wizard README](../etc/scripts/ce-properties-wizard/README.md).
## Manual configuration
If you need more control or want to understand how the configuration works, read on for the manual approach.
### Configuration
Compiler configuration is done through the `etc/config/c++.*.properties` files (for C++, other languages follow the Compiler configuration is done through the `etc/config/c++.*.properties` files (for C++, other languages follow the
obvious pattern, replace as needed for your case). obvious pattern, replace as needed for your case).
@@ -84,9 +131,9 @@ forward if that group is redefined in a higher-priority configuration file (e.g.
The `compilerType` option is special: it refers to the Javascript class in `lib/compilers/*.ts` which handles running The `compilerType` option is special: it refers to the Javascript class in `lib/compilers/*.ts` which handles running
and handling output for this compiler type. and handling output for this compiler type.
## Adding a new compiler locally ## Adding a new compiler manually
It should be pretty straightforward to add a compiler of your own. Create a `etc/config/c++.local.properties` file and If the wizard doesn't work for your use case or you need fine-grained control, you can manually add a compiler. Create a `etc/config/c++.local.properties` file and
override the `compilers` list to include your own compiler, and its configuration. override the `compilers` list to include your own compiler, and its configuration.
Once you've done that, running `make` should pick up the configuration and during startup you should see your compiler Once you've done that, running `make` should pick up the configuration and during startup you should see your compiler

View File

@@ -0,0 +1,283 @@
# CE Properties Wizard
An interactive command-line tool for adding custom compilers to your local Compiler Explorer installation.
## Features
- **Automatic Detection**: Detects compiler type and language from the executable path
- **Auto-Discovery**: Automatically finds and adds all compilers in your PATH
- **Interactive Mode**: Guided prompts for configuration
- **Automation Support**: Command-line flags for scripting
- **Group Management**: Automatically adds compilers to appropriate groups
- **Validation**: Validates generated properties with `propscheck.py`
- **Safe Updates**: Only adds/updates, never removes existing configurations
## Requirements
The wizard requires Python 3.10+ and Poetry. The run scripts handle all setup automatically.
## Usage
### Interactive Mode
Run without arguments for a fully interactive experience:
**Linux/macOS:**
```bash
./run.sh
```
**Windows:**
```powershell
.\run.ps1
```
### Path-First Mode
Provide a compiler path to skip the first prompt:
**Linux/macOS:**
```bash
./run.sh /usr/local/bin/g++-13
```
**Windows:**
```powershell
.\run.ps1 "C:\MinGW\bin\g++.exe"
```
### Automated Mode
Use command-line flags to automate the process:
**Linux/macOS:**
```bash
./run.sh /usr/local/bin/g++-13 --yes
```
**Windows:**
```powershell
.\run.ps1 "C:\MinGW\bin\g++.exe" --yes
```
### Full Automation Example
**Linux/macOS:**
```bash
./run.sh /path/to/compiler \
--id custom-gcc-13 \
--name "GCC 13.2.0" \
--group gcc \
--options "-std=c++20" \
--language c++ \
--yes
```
**Windows:**
```powershell
.\run.ps1 "C:\path\to\compiler.exe" `
--id custom-gcc-13 `
--name "GCC 13.2.0" `
--group gcc `
--options "-std=c++20" `
--language c++ `
--yes
```
### Auto-Discovery
Automatically discover and add all compilers in your PATH:
```bash
./auto_discover_compilers.py --dry-run # Preview what would be found
./auto_discover_compilers.py --languages c++,rust # Add only C++ and Rust compilers
./auto_discover_compilers.py --yes # Add all found compilers automatically
```
### Batch Processing
Add multiple compilers with a simple loop:
**Linux/macOS:**
```bash
for compiler in /opt/compilers/*/bin/*; do
./run.sh "$compiler" --yes
done
```
**Windows:**
```powershell
Get-ChildItem "C:\Compilers\*\bin\*.exe" | ForEach-Object {
.\run.ps1 $_.FullName --yes
}
```
## Command-Line Options
- `COMPILER_PATH`: Path to the compiler executable (optional in interactive mode)
- `--id`: Compiler ID (auto-generated if not specified)
- `--name`: Display name for the compiler
- `--group`: Compiler group to add to (e.g., gcc, clang)
- `--options`: Default compiler options
- `--language`: Programming language (auto-detected if not specified)
- `--yes, -y`: Skip confirmation prompts
- `--non-interactive`: Run in non-interactive mode with auto-detected values
- `--config-dir`: Path to etc/config directory (auto-detected if not specified)
- `--verify-only`: Only detect and display compiler information without making changes
- `--list-types`: List all supported compiler types and exit
- `--reorganize LANGUAGE`: Reorganize an existing properties file for the specified language
- `--validate-discovery`: Run discovery validation to verify the compiler is detected (default for local environment)
- `--env ENV`: Environment to target (local, amazon, etc.) - defaults to 'local'
## Supported Languages
The wizard currently supports:
**Systems Languages:**
- C++, C, CUDA
- Rust, Zig, V, Odin
- Carbon, Mojo
**Popular Compiled Languages:**
- D (DMD, LDC, GDC)
- Swift, Nim, Crystal
- Go, Kotlin, Java
**Functional Languages:**
- Haskell (GHC)
- OCaml, Scala
**.NET Languages:**
- C#, F#
**Scripting/Dynamic Languages:**
- Python, Ruby, Julia
- Dart, Elixir, Erlang
**Other Languages:**
- Fortran, Pascal, Ada
- COBOL, Assembly (NASM, GAS, YASM)
## Compiler Detection
The wizard attempts to detect compiler type by running version commands:
- GCC: `--version`
- Clang: `--version`
- Intel: `--version`
- MSVC: `/help`
- NVCC: `--version`
- Rust: `--version`
- Go: `version`
- Python: `--version`
If detection fails, you can manually specify the compiler type.
## Configuration Files
The wizard modifies `<language>.local.properties` files in `etc/config/`. It:
- Preserves existing content and formatting
- Creates backup files before modification
- Adds compilers to groups by default
- Ensures unique compiler IDs
## Examples
### Add a custom GCC installation
**Linux/macOS:**
```bash
./run.sh /opt/gcc-13.2.0/bin/g++
```
**Windows:**
```powershell
.\run.ps1 "C:\TDM-GCC-64\bin\g++.exe"
```
### Add a cross-compiler
**Linux/macOS:**
```bash
./run.sh /usr/bin/arm-linux-gnueabihf-g++ \
--name "ARM GCC 11.2" \
--group arm-gcc \
--yes
```
**Windows:**
```powershell
.\run.ps1 "C:\arm-toolchain\bin\arm-none-eabi-g++.exe" `
--name "ARM GCC 11.2" `
--group arm-gcc `
--yes
```
### Add a Python interpreter
**Linux/macOS:**
```bash
./run.sh /usr/local/bin/python3.12 --yes
```
**Windows:**
```powershell
.\run.ps1 "C:\Python312\python.exe" --yes
```
### Verify compiler detection only
**Linux/macOS:**
```bash
./run.sh /usr/bin/g++-13 --verify-only
```
**Windows:**
```powershell
.\run.ps1 "C:\MinGW\bin\g++.exe" --verify-only
```
### List all supported compiler types
**Linux/macOS:**
```bash
./run.sh --list-types
```
**Windows:**
```powershell
.\run.ps1 --list-types
```
This will output something like:
```
Detected compiler information:
Path: /usr/bin/g++-13
Language: C++
Compiler Type: gcc
Version: 13.2.0
Semver: 13.2.0
Suggested ID: custom-gcc-13-2-0
Suggested Name: GCC 13.2.0
Suggested Group: gcc
```
## Troubleshooting
### Compiler not detected
If the wizard can't detect your compiler type, it will prompt you to select one manually.
### Permission errors
Ensure you have write permissions to the `etc/config` directory.
### Validation failures
If `propscheck.py` reports errors, check the generated properties file for syntax issues.
## Development
To contribute to the wizard:
1. Format code: `./run.sh --format`
2. Check formatting: `./run.sh --format --check`
3. Run tests: `poetry run pytest` (after `poetry install`)
The `--format` flag runs black, ruff, and pytype formatters on the codebase.

View File

@@ -0,0 +1,284 @@
#!/usr/bin/env python3
"""
CE Compiler Auto-Discovery Tool
Automatically discovers compilers in PATH directories and adds them using
the CE Properties Wizard.
"""
import argparse
import os
import subprocess
import sys
from pathlib import Path
from typing import Dict, List, Set
# Compiler patterns for each language
COMPILER_PATTERNS = {
'c++': ['g++', 'g++-*', 'clang++', 'clang++-*', 'icpc*', 'icx*'],
'c': ['gcc', 'gcc-[0-9]*', 'clang', 'clang-[0-9]*', 'icc*', 'cc'],
'cuda': ['nvcc*'],
'rust': ['rustc*'],
'go': ['go', 'gccgo*'],
'python': ['python*', 'python3*', 'pypy*'],
'java': ['javac*', 'java'],
'fortran': ['gfortran*', 'ifort*', 'ifx*'],
'pascal': ['fpc'],
'kotlin': ['kotlin*', 'kotlinc*'],
'zig': ['zig'],
'dart': ['dart'],
'd': ['dmd*', 'ldc*', 'ldc2*', 'gdc*'],
'swift': ['swift*', 'swiftc*'],
'nim': ['nim'],
'crystal': ['crystal'],
'v': ['v'],
'haskell': ['ghc*'],
'ocaml': ['ocaml*'],
'scala': ['scala*', 'scalac*'],
'csharp': ['csc*', 'mcs*', 'dotnet'],
'fsharp': ['fsharpc*', 'dotnet'],
'ruby': ['ruby*'],
'julia': ['julia'],
'elixir': ['elixir*'],
'erlang': ['erlc*', 'erl'],
'assembly': ['nasm*', 'yasm*', 'as'],
'carbon': ['carbon*'],
'mojo': ['mojo*'],
'odin': ['odin*'],
'ada': ['gnatmake*', 'gprbuild*', 'gnat*'],
'cobol': ['cobc*', 'gnucobol*', 'gcobol*'],
}
# Default exclude patterns
DEFAULT_EXCLUDES = {
'wrapper', 'distcc', 'ccache', '-config', 'config-',
'-ar', '-nm', '-ranlib', '-strip', 'filt', 'format',
'calls', 'flow', 'stat', '-gdb', 'argcomplete', 'build',
'ldconfig', 'ldconfig.real', '-bpfcc', 'bpfcc', 'scalar',
'pythongc-bpfcc', 'pythonflow-bpfcc', 'pythoncalls-bpfcc', 'pythonstat-bpfcc'
}
def get_path_dirs() -> List[Path]:
"""Get all directories from PATH environment variable."""
path = os.environ.get('PATH', '')
return [Path(p) for p in path.split(':') if p.strip()]
def should_exclude(name: str, excludes: Set[str]) -> bool:
"""Check if a compiler name should be excluded."""
return any(exclude in name for exclude in excludes)
def find_compilers_in_dir(directory: Path, patterns: List[str], excludes: Set[str]) -> List[Path]:
"""Find compilers matching patterns in a directory."""
compilers = []
if not directory.exists() or not directory.is_dir():
return compilers
for pattern in patterns:
# Simple glob matching
for compiler in directory.glob(pattern):
if (compiler.is_file() or compiler.is_symlink()) and \
os.access(compiler, os.X_OK) and \
not should_exclude(compiler.name, excludes):
compilers.append(compiler)
return compilers
def resolve_duplicates(compilers: List[Path]) -> List[Path]:
"""Remove duplicate compilers (same resolved path)."""
seen = set()
unique_compilers = []
for compiler in compilers:
try:
resolved = compiler.resolve()
if resolved not in seen:
seen.add(resolved)
unique_compilers.append(compiler)
except (OSError, RuntimeError):
# If we can't resolve, keep the original
unique_compilers.append(compiler)
return unique_compilers
def discover_compilers(languages: List[str], search_dirs: List[Path] = None,
excludes: Set[str] = None) -> Dict[str, List[Path]]:
"""Discover compilers for specified languages."""
if search_dirs is None:
search_dirs = get_path_dirs()
if excludes is None:
excludes = DEFAULT_EXCLUDES
discovered = {}
for language in languages:
if language not in COMPILER_PATTERNS:
print(f"Warning: Unknown language '{language}'", file=sys.stderr)
continue
patterns = COMPILER_PATTERNS[language]
compilers = []
for directory in search_dirs:
compilers.extend(find_compilers_in_dir(directory, patterns, excludes))
if compilers:
# Remove duplicates and sort
unique_compilers = resolve_duplicates(compilers)
discovered[language] = sorted(unique_compilers, key=lambda x: x.name)
return discovered
def add_compiler_with_wizard(compiler: Path, language: str, script_dir: Path,
wizard_args: List[str], dry_run: bool) -> bool:
"""Add a compiler using the CE Properties Wizard."""
if dry_run:
return True
cmd = [
str(script_dir / 'run.sh'),
str(compiler),
'--yes',
'--language', language
] + wizard_args
try:
result = subprocess.run(cmd, capture_output=True, text=True)
return result.returncode == 0
except Exception:
return False
def main():
parser = argparse.ArgumentParser(
description='CE Compiler Auto-Discovery Tool',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
%(prog)s # Interactive discovery of all languages
%(prog)s --dry-run # Preview what would be discovered
%(prog)s --languages c++,rust,go # Only discover C++, Rust, and Go
%(prog)s --yes --languages c++,c # Non-interactive C/C++ discovery
""")
parser.add_argument('--languages',
help='Comma-separated list of languages to discover (default: all)')
parser.add_argument('--search-dirs',
help='Colon-separated search directories (default: PATH dirs)')
parser.add_argument('--exclude',
help='Comma-separated exclude patterns')
parser.add_argument('--dry-run', action='store_true',
help='Show what would be added without making changes')
parser.add_argument('--yes', '-y', action='store_true',
help='Skip confirmation prompts')
parser.add_argument('--config-dir',
help='Path to etc/config directory')
parser.add_argument('--env', default='local',
help='Environment to target (local, amazon, etc.)')
args = parser.parse_args()
# Get script directory
script_dir = Path(__file__).parent
wizard_script = script_dir / 'run.sh'
if not wizard_script.exists():
print(f"Error: CE Properties Wizard not found at {wizard_script}", file=sys.stderr)
sys.exit(1)
# Parse languages
if args.languages:
languages = [lang.strip() for lang in args.languages.split(',')]
else:
languages = list(COMPILER_PATTERNS.keys())
# Parse search directories
search_dirs = None
if args.search_dirs:
search_dirs = [Path(d) for d in args.search_dirs.split(':') if d.strip()]
# Parse excludes
excludes = DEFAULT_EXCLUDES.copy()
if args.exclude:
excludes.update(args.exclude.split(','))
# Discover compilers
print("CE Compiler Auto-Discovery Tool")
print("=" * 35)
print()
if args.dry_run:
print("DRY RUN MODE - No compilers will actually be added")
print()
discovered = discover_compilers(languages, search_dirs, excludes)
if not discovered:
print("No compilers found matching the specified criteria")
sys.exit(1)
# Show results
total_count = sum(len(compilers) for compilers in discovered.values())
print(f"Found {total_count} compilers:")
print()
for language, compilers in discovered.items():
print(f"{language.upper()} ({len(compilers)} compilers):")
for compiler in compilers:
print(f"{compiler}")
print()
# Confirm before adding
if not args.dry_run and not args.yes:
response = input("Add these compilers? [y/N] ")
if not response.lower().startswith('y'):
print("Operation cancelled")
sys.exit(0)
if args.dry_run:
print("Dry run complete - no changes made")
sys.exit(0)
# Add compilers
print("Adding compilers using CE Properties Wizard...")
print()
wizard_args = []
if args.config_dir:
wizard_args.extend(['--config-dir', args.config_dir])
if args.env != 'local':
wizard_args.extend(['--env', args.env])
added_count = 0
failed_count = 0
for language, compilers in discovered.items():
for compiler in compilers:
print(f"Adding {compiler} ({language})...", end=' ')
if add_compiler_with_wizard(compiler, language, script_dir, wizard_args, args.dry_run):
print("")
added_count += 1
else:
print("")
failed_count += 1
print()
print("Summary:")
print(f" ✓ Successfully added: {added_count} compilers")
if failed_count > 0:
print(f" ✗ Failed to add: {failed_count} compilers")
print()
print("Auto-discovery complete!")
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,3 @@
"""CE Properties Wizard - Interactive tool for adding compilers to Compiler Explorer."""
__version__ = "0.1.0"

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,541 @@
"""Main CLI entry point for CE Properties Wizard."""
import os
import shlex
import sys
from pathlib import Path
from typing import Optional
import click
import inquirer
from colorama import Fore, Style, init
from .compiler_detector import LANGUAGE_CONFIGS, CompilerDetector, get_supported_compiler_types
from .config_manager import ConfigManager
from .models import CompilerInfo
from .utils import find_ce_config_directory
# Initialize colorama for cross-platform color support
init(autoreset=True)
def print_success(message: str):
"""Print success message in green."""
click.echo(f"{Fore.GREEN}{message}{Style.RESET_ALL}")
def print_error(message: str):
"""Print error message in red."""
click.echo(f"{Fore.RED}{message}{Style.RESET_ALL}", err=True)
def print_info(message: str):
"""Print info message in blue."""
click.echo(f"{Fore.BLUE} {message}{Style.RESET_ALL}")
def print_warning(message: str):
"""Print warning message in yellow."""
click.echo(f"{Fore.YELLOW}{message}{Style.RESET_ALL}")
def format_compiler_options(options_input: str) -> str:
"""Format compiler options properly.
Takes space-separated options and quotes any that contain spaces.
Args:
options_input: Raw options string from user input
Returns:
Properly formatted options string with quoted options containing spaces
"""
if not options_input or not options_input.strip():
return ""
# Split by spaces but respect quoted strings
try:
options = shlex.split(options_input)
except ValueError:
# If shlex fails (unmatched quotes), fall back to simple split
options = options_input.split()
# Format each option - quote it if it contains spaces
formatted_options = []
for opt in options:
opt = opt.strip()
if opt:
if " " in opt and not (opt.startswith('"') and opt.endswith('"')):
formatted_options.append(f'"{opt}"')
else:
formatted_options.append(opt)
return " ".join(formatted_options)
@click.command()
@click.argument("compiler_path", required=False)
@click.option("--id", "compiler_id", help="Compiler ID (auto-generated if not specified)")
@click.option("--name", "display_name", help="Display name for the compiler")
@click.option("--group", help="Compiler group to add to")
@click.option("--options", help="Default compiler options")
@click.option("--language", help="Programming language (auto-detected if not specified)")
@click.option("--yes", "-y", is_flag=True, help="Skip confirmation prompts")
@click.option("--non-interactive", is_flag=True, help="Run in non-interactive mode with auto-detected values")
@click.option("--config-dir", type=click.Path(exists=True), help="Path to etc/config directory")
@click.option("--verify-only", is_flag=True, help="Only detect and display compiler information without making changes")
@click.option("--list-types", is_flag=True, help="List all supported compiler types and exit")
@click.option("--reorganize", help="Reorganize an existing properties file for the specified language")
@click.option(
"--validate-discovery",
is_flag=True,
help="Run discovery validation to verify the compiler is detected (default for local environment)",
)
@click.option("--env", default="local", help="Environment to target (local, amazon, etc.)")
@click.option("--debug", is_flag=True, help="Enable debug output including subprocess commands")
def cli(
compiler_path: Optional[str],
compiler_id: Optional[str],
display_name: Optional[str],
group: Optional[str],
options: Optional[str],
language: Optional[str],
yes: bool,
non_interactive: bool,
config_dir: Optional[str],
verify_only: bool,
list_types: bool,
reorganize: Optional[str],
validate_discovery: bool,
env: str,
debug: bool,
):
"""CE Properties Wizard - Add compilers to your Compiler Explorer installation.
Examples:
ce-props-wizard # Interactive mode (local environment)
ce-props-wizard /usr/bin/g++-13 # Path-first mode
ce-props-wizard /usr/bin/g++-13 --yes # Automated mode
ce-props-wizard --env amazon /usr/bin/g++ # Target amazon environment
ce-props-wizard --list-types # List all supported compiler types
ce-props-wizard /usr/bin/g++ --verify-only # Just detect compiler info
"""
# Handle --list-types flag
if list_types:
try:
supported_types = get_supported_compiler_types()
click.echo(f"Found {len(supported_types)} supported compiler types:\n")
for compiler_type in sorted(supported_types):
click.echo(compiler_type)
sys.exit(0)
except Exception as e:
print_error(f"Error reading compiler types: {e}")
sys.exit(1)
# Handle --reorganize flag
if reorganize:
try:
# Find config directory
if config_dir:
config_mgr = ConfigManager(Path(config_dir), env, debug=debug)
else:
config_mgr = ConfigManager(find_ce_config_directory(), env, debug=debug)
print_info(f"Reorganizing {reorganize} properties file...")
# Check if language is valid
if reorganize not in LANGUAGE_CONFIGS:
print_error(f"Unknown language: {reorganize}")
print_info(f"Available languages: {', '.join(LANGUAGE_CONFIGS.keys())}")
sys.exit(1)
file_path = config_mgr.get_properties_path(reorganize)
if not file_path.exists():
print_error(f"No {env} properties file found for {reorganize}: {file_path}")
sys.exit(1)
config_mgr.reorganize_existing_file(reorganize)
print_success(f"Reorganized {file_path}")
# Validate with propscheck
print_info("Validating with propscheck.py...")
valid, message = config_mgr.validate_with_propscheck(reorganize)
if valid:
print_success(message)
else:
print_error(message)
sys.exit(0)
except Exception as e:
print_error(f"Error reorganizing file: {e}")
sys.exit(1)
# Skip banner in verify-only mode
if not verify_only:
click.echo(f"{Fore.CYAN}{'='*60}{Style.RESET_ALL}")
click.echo(f"{Fore.CYAN}Compiler Explorer Properties Wizard{Style.RESET_ALL}")
click.echo(f"{Fore.CYAN}{'='*60}{Style.RESET_ALL}\n")
try:
# Find config directory only if needed
if not verify_only:
if config_dir:
config_path = Path(config_dir)
else:
config_path = find_ce_config_directory()
print_info(f"Using config directory: {config_path}")
print_info(f"Targeting environment: {env}")
config_mgr = ConfigManager(config_path, env, debug=debug)
else:
config_mgr = None
# Initialize detector
detector = CompilerDetector(debug=debug)
# Get compiler path if not provided
if not compiler_path:
questions = [
inquirer.Text(
"compiler_path",
message="Enter the full path to the compiler executable",
validate=lambda _, x: os.path.isfile(x),
)
]
answers = inquirer.prompt(questions)
if not answers:
print_error("Cancelled by user")
sys.exit(1)
compiler_path = answers["compiler_path"]
# Validate compiler path
compiler_path = os.path.abspath(compiler_path)
if not os.path.isfile(compiler_path):
print_error(f"Compiler not found: {compiler_path}")
sys.exit(1)
if not os.access(compiler_path, os.X_OK):
print_error(f"File is not executable: {compiler_path}")
sys.exit(1)
# Detect compiler information
print_info("Detecting compiler type and language...")
try:
detected_info = detector.detect_from_path(compiler_path)
if detected_info.compiler_type:
print_success(f"Detected: {detected_info.name} ({LANGUAGE_CONFIGS[detected_info.language].name})")
else:
print_warning("Could not detect compiler type")
except Exception as e:
print_error(f"Detection failed: {e}")
# Create minimal info
detected_info = CompilerInfo(
id="custom-compiler",
name=os.path.basename(compiler_path),
exe=compiler_path,
language=language or "c++",
)
# Override with command-line options
if language:
detected_info.language = language
# Suggest appropriate group if not already set
if not detected_info.group:
if not verify_only and config_mgr is not None:
# Normal mode - create config manager and suggest group
suggested_group = config_mgr.suggest_appropriate_group(detected_info)
if suggested_group:
detected_info.group = suggested_group
else:
# Verify-only mode - create a temporary config manager just for suggestion
temp_config_mgr = ConfigManager(find_ce_config_directory(), env, debug=debug)
suggested_group = temp_config_mgr.suggest_appropriate_group(detected_info)
if suggested_group:
detected_info.group = suggested_group
# Initialize flag for forcing custom ID/name
force_custom_id_name = False
# Check for existing compiler by path early (before prompts)
if not verify_only and config_mgr is not None:
existing_compiler_id = config_mgr.check_existing_compiler_by_path(compiler_path, detected_info.language)
if existing_compiler_id:
file_path = config_mgr.get_properties_path(detected_info.language)
print_warning(f"Compiler already exists in {env} environment!")
print_info(f"Existing compiler ID: {existing_compiler_id}")
print_info(f"Executable path: {compiler_path}")
print_info(f"Properties file: {file_path}")
# If automated mode (-y), exit immediately
if yes or non_interactive:
print_info("No changes were made.")
sys.exit(0)
# In interactive mode, ask if user wants to continue with different ID/name
if not click.confirm("\nWould you like to add this compiler anyway with a different ID and name?"):
print_info("No changes were made.")
sys.exit(0)
print_info("You will need to provide a unique compiler ID and custom name.")
# Set flag to force custom ID and name prompts
force_custom_id_name = True
# Suggest the group from the existing duplicate compiler
if config_mgr is not None:
suggested_group = config_mgr.suggest_appropriate_group(detected_info, existing_compiler_id)
if suggested_group and not detected_info.group:
detected_info.group = suggested_group
# If verify-only mode, display info and exit
if verify_only:
click.echo("\nDetected compiler information:")
click.echo(f" Path: {compiler_path}")
lang_name = (
LANGUAGE_CONFIGS[detected_info.language].name
if detected_info.language in LANGUAGE_CONFIGS
else detected_info.language
)
click.echo(f" Language: {lang_name}")
click.echo(f" Compiler Type: {detected_info.compiler_type or 'unknown'}")
click.echo(f" Version: {detected_info.version or 'unknown'}")
click.echo(f" Semver: {detected_info.semver or 'unknown'}")
if detected_info.target:
click.echo(f" Target: {detected_info.target}")
click.echo(f" Cross-compiler: {'Yes' if detected_info.is_cross_compiler else 'No'}")
click.echo(f" Suggested ID: {detected_info.id}")
click.echo(f" Suggested Name: {detected_info.name}")
click.echo(f" Suggested Group: {detected_info.group or 'none'}")
sys.exit(0)
# Interactive prompts for missing information
if not yes and not non_interactive:
questions = []
# Windows SDK path prompt for MSVC compilers if auto-detection failed
if detected_info.needs_sdk_prompt:
print_info("Windows SDK auto-detection failed. You can optionally specify the Windows SDK path.")
print_info("Example: Z:/compilers/windows-kits-10 (leave empty to skip)")
sdk_question = inquirer.Text(
"windows_sdk_path",
message="Windows SDK base path (optional)",
default="",
validate=lambda _, x: x == "" or os.path.isdir(x.replace("\\", "/"))
)
sdk_answers = inquirer.prompt([sdk_question])
if sdk_answers and sdk_answers["windows_sdk_path"].strip():
# Apply the user-provided SDK path
detected_info = detector.set_windows_sdk_path(detected_info, sdk_answers["windows_sdk_path"].strip())
print_success(f"Windows SDK paths added from: {sdk_answers['windows_sdk_path']}")
# Language selection if needed
if not language and detected_info.language:
lang_choices = [(LANGUAGE_CONFIGS[k].name, k) for k in LANGUAGE_CONFIGS.keys()]
questions.append(
inquirer.List(
"language", message="Programming language", choices=lang_choices, default=detected_info.language
)
)
# Compiler ID - force custom if duplicate exists
if force_custom_id_name:
questions.append(
inquirer.Text(
"compiler_id",
message="Compiler ID (must be unique)",
default=compiler_id or "",
validate=lambda _, x: bool(x and x.strip() and x != detected_info.id),
)
)
else:
questions.append(
inquirer.Text(
"compiler_id",
message="Compiler ID",
default=compiler_id or detected_info.id,
validate=lambda _, x: bool(x and x.strip()),
)
)
# Display name - force custom if duplicate exists
if force_custom_id_name:
questions.append(
inquirer.Text(
"display_name",
message="Display name (must be custom)",
default=display_name or "",
validate=lambda _, x: bool(x and x.strip() and x != detected_info.name),
)
)
else:
questions.append(
inquirer.Text("display_name", message="Display name", default=display_name or detected_info.name)
)
# Compiler type (if not detected)
if not detected_info.compiler_type:
# Get all supported compiler types dynamically
supported_types = sorted(get_supported_compiler_types())
# Add 'other' as fallback option
type_choices = supported_types + ["other"]
questions.append(
inquirer.List("compiler_type", message="Compiler type", choices=type_choices, default="other")
)
# Group
questions.append(
inquirer.Text(
"group",
message="Add to group",
default=group or detected_info.group or detected_info.compiler_type or "",
)
)
# Options
questions.append(
inquirer.Text(
"options",
message="Additional options (space-separated, quote options with spaces)",
default=options or "",
)
)
if questions:
answers = inquirer.prompt(questions)
if not answers:
print_error("Cancelled by user")
sys.exit(1)
# Update detected info
if "language" in answers:
detected_info.language = answers["language"]
if "compiler_id" in answers:
detected_info.id = answers["compiler_id"]
if "display_name" in answers:
detected_info.name = answers["display_name"]
# If this is a duplicate override scenario, force the name to be included
if force_custom_id_name:
detected_info.force_name = True
if "compiler_type" in answers:
compiler_type = answers["compiler_type"]
# Validate compiler type against supported types
if compiler_type != "other":
supported_types = get_supported_compiler_types()
if compiler_type not in supported_types:
print_warning(f"'{compiler_type}' is not a recognized compiler type in Compiler Explorer")
detected_info.compiler_type = compiler_type
if "group" in answers and answers["group"]:
detected_info.group = answers["group"]
if "options" in answers and answers["options"]:
detected_info.options = format_compiler_options(answers["options"])
else:
# In automated mode, use command-line values
if compiler_id:
detected_info.id = compiler_id
if display_name:
detected_info.name = display_name
# If this is a duplicate override scenario, force the name to be included
if force_custom_id_name:
detected_info.force_name = True
if group:
detected_info.group = group
if options:
detected_info.options = format_compiler_options(options)
# Ensure unique ID (config_mgr should not be None at this point)
assert config_mgr is not None, "config_mgr should not be None in non-verify mode"
original_id = detected_info.id
detected_info.id = config_mgr.ensure_compiler_id_unique(detected_info.id, detected_info.language)
if detected_info.id != original_id:
print_warning(f"ID already exists, using: {detected_info.id}")
# Show configuration preview
print_info("\nConfiguration preview:")
normalized_exe_path = detected_info.exe.replace("\\", "/")
click.echo(f" compiler.{detected_info.id}.exe={normalized_exe_path}")
# Check if semver will be available (either detected or extracted)
semver_to_use = detected_info.semver
if not semver_to_use:
# Try to extract version like the config manager will do
try:
semver_to_use = config_mgr._extract_compiler_version(detected_info.exe)
except Exception:
pass
# Show semver if available
if semver_to_use:
click.echo(f" compiler.{detected_info.id}.semver={semver_to_use}")
# Show name if semver is not available OR if this is a duplicate override scenario
if detected_info.name and (not semver_to_use or force_custom_id_name):
click.echo(f" compiler.{detected_info.id}.name={detected_info.name}")
if detected_info.compiler_type:
click.echo(f" compiler.{detected_info.id}.compilerType={detected_info.compiler_type}")
if detected_info.options:
click.echo(f" compiler.{detected_info.id}.options={detected_info.options}")
if detected_info.java_home:
click.echo(f" compiler.{detected_info.id}.java_home={detected_info.java_home}")
if detected_info.runtime:
click.echo(f" compiler.{detected_info.id}.runtime={detected_info.runtime}")
if detected_info.execution_wrapper:
click.echo(f" compiler.{detected_info.id}.executionWrapper={detected_info.execution_wrapper}")
if detected_info.include_path:
click.echo(f" compiler.{detected_info.id}.includePath={detected_info.include_path}")
if detected_info.lib_path:
click.echo(f" compiler.{detected_info.id}.libPath={detected_info.lib_path}")
if detected_info.group:
click.echo(f" Will add to group: {detected_info.group}")
# Confirm
file_path = config_mgr.get_properties_path(detected_info.language)
if not yes and not non_interactive:
if not click.confirm(f"\nUpdate {file_path}?"):
print_error("Cancelled by user")
sys.exit(1)
# Add compiler
config_mgr.add_compiler(detected_info)
print_success("Configuration updated successfully!")
# Validate with propscheck
print_info("Validating with propscheck.py...")
valid, message = config_mgr.validate_with_propscheck(detected_info.language)
if valid:
print_success(message)
else:
print_error(message)
# Don't exit with error, as the file was written successfully
# Discovery validation (default for local environment, optional for others)
should_validate_discovery = validate_discovery or (env == "local")
if should_validate_discovery:
print_info("Validating with discovery...")
valid, message, discovered_semver = config_mgr.validate_with_discovery(
detected_info.language, detected_info.id
)
if valid:
print_success(message)
if discovered_semver:
print_info(f"Discovered semver: {discovered_semver}")
else:
print_error(message)
print_info(
"Note: Discovery validation failed, but the compiler was added to the properties file successfully."
)
click.echo(f"\n{Fore.GREEN}Compiler added successfully!{Style.RESET_ALL}")
click.echo("You may need to restart Compiler Explorer for changes to take effect.")
except KeyboardInterrupt:
print_error("\nCancelled by user")
sys.exit(1)
except Exception as e:
print_error(f"Error: {e}")
sys.exit(1)
if __name__ == "__main__":
cli()

View File

@@ -0,0 +1,54 @@
"""Data models for the CE Properties Wizard."""
import re
from typing import List, Optional
from pydantic import BaseModel, Field, validator
class CompilerInfo(BaseModel):
"""Model representing compiler information."""
id: str = Field(..., description="Unique identifier for the compiler")
name: str = Field(..., description="Display name for the compiler")
exe: str = Field(..., description="Path to the compiler executable")
compiler_type: Optional[str] = Field(None, description="Type of compiler (gcc, clang, etc)")
version: Optional[str] = Field(None, description="Compiler version")
semver: Optional[str] = Field(None, description="Semantic version")
group: Optional[str] = Field(None, description="Compiler group to add to")
options: Optional[str] = Field(None, description="Default compiler options")
language: str = Field(..., description="Programming language")
target: Optional[str] = Field(None, description="Target platform (for cross-compilers)")
is_cross_compiler: bool = Field(False, description="Whether this is a cross-compiler")
force_name: bool = Field(False, description="Force inclusion of .name property even when semver exists")
java_home: Optional[str] = Field(None, description="JAVA_HOME path for Java-based compilers")
runtime: Optional[str] = Field(None, description="Runtime executable path for Java-based compilers")
execution_wrapper: Optional[str] = Field(None, description="Execution wrapper path for languages like Dart")
include_path: Optional[str] = Field(None, description="Include paths for MSVC compilers")
lib_path: Optional[str] = Field(None, description="Library paths for MSVC compilers")
needs_sdk_prompt: bool = Field(False, description="Whether to prompt user for Windows SDK path")
@validator("id")
def validate_id(cls, value): # noqa: N805
"""Ensure ID is valid for properties files."""
if not re.match(r"^[a-zA-Z0-9_-]+$", value):
raise ValueError("ID must contain only alphanumeric characters, hyphens, and underscores")
return value
class LanguageConfig(BaseModel):
"""Model representing language configuration."""
name: str = Field(..., description="Language name")
properties_file: str = Field(..., description="Properties filename (without path, defaults to local)")
compiler_types: List[str] = Field(default_factory=list, description="Known compiler types for this language")
extensions: List[str] = Field(default_factory=list, description="File extensions")
keywords: List[str] = Field(default_factory=list, description="Keywords in compiler path/name")
def get_properties_file(self, env: str = "local") -> str:
"""Get properties file name for specified environment."""
if env == "local":
return self.properties_file
else:
# Replace .local. with .{env}.
return self.properties_file.replace(".local.", f".{env}.")

View File

@@ -0,0 +1,608 @@
"""Surgical properties file editor that preserves existing structure."""
import re
from pathlib import Path
from typing import List, Optional, Tuple
from .models import CompilerInfo
from .utils import create_backup
class PropertiesFileEditor:
"""Surgical editor that makes minimal changes to properties files."""
def __init__(self, file_path: Path):
self.file_path = file_path
self.lines: List[str] = []
self.load_file()
def load_file(self):
"""Load file content, preserving all structure."""
if self.file_path.exists():
with open(self.file_path, "r", encoding="utf-8") as f:
self.lines = [line.rstrip("\n") for line in f.readlines()]
else:
self.lines = []
def save_file(self):
"""Save file with minimal changes."""
# Create backup if file exists
if self.file_path.exists():
create_backup(self.file_path)
with open(self.file_path, "w", encoding="utf-8") as f:
for line in self.lines:
f.write(f"{line}\n")
def find_compilers_line(self) -> Optional[int]:
"""Find the compilers= line."""
for i, line in enumerate(self.lines):
if line.startswith("compilers="):
return i
return None
def find_group_section(self, group_name: str) -> Tuple[Optional[int], Optional[int]]:
"""Find the start and end of a group section.
Returns:
(start_line, end_line) where start_line is the first group.{name}. line
and end_line is the line before the next group or compiler section starts.
"""
start_line = None
end_line = None
# Find start of this group
group_prefix = f"group.{group_name}."
for i, line in enumerate(self.lines):
if line.startswith(group_prefix):
start_line = i
break
if start_line is None:
return None, None
# Find end of this group (before next group or compiler section)
for i in range(start_line + 1, len(self.lines)):
line = self.lines[i].strip()
if (
line.startswith("group.")
and not line.startswith(group_prefix)
or line.startswith("compiler.")
or line.startswith("libs=")
or line.startswith("tools=")
or line.startswith("#")
and ("####" in line or "Installed" in line)
):
end_line = i
break
if end_line is None:
end_line = len(self.lines)
return start_line, end_line
def find_compiler_section(self, compiler_id: str) -> Tuple[Optional[int], Optional[int]]:
"""Find the start and end of a compiler section."""
start_line = None
end_line = None
# Find start of this compiler
compiler_prefix = f"compiler.{compiler_id}."
for i, line in enumerate(self.lines):
if line.startswith(compiler_prefix):
start_line = i
break
if start_line is None:
return None, None
# Find end of this compiler (before next compiler, group, or libs/tools section)
for i in range(start_line + 1, len(self.lines)):
line = self.lines[i].strip()
if (
line.startswith("compiler.")
and not line.startswith(compiler_prefix)
or line.startswith("group.")
or line.startswith("libs=")
or line.startswith("tools=")
or line.startswith("#")
and ("####" in line or "Installed" in line)
):
end_line = i
break
if end_line is None:
end_line = len(self.lines)
return start_line, end_line
def get_existing_groups_from_compilers_line(self) -> List[str]:
"""Extract group names from the compilers= line."""
compilers_line_idx = self.find_compilers_line()
if compilers_line_idx is None:
return []
line = self.lines[compilers_line_idx]
# Extract groups from compilers=&group1:&group2:...
if "=" in line:
value = line.split("=", 1)[1]
groups = []
for part in value.split(":"):
part = part.strip()
if part.startswith("&"):
groups.append(part[1:]) # Remove & prefix
return groups
return []
def add_group_to_compilers_line(self, group_name: str):
"""Add a group to the compilers= line if not already present."""
existing_groups = self.get_existing_groups_from_compilers_line()
if group_name in existing_groups:
return # Already exists
# Check if this group is referenced by any existing parent groups
# (e.g., vcpp_x64 might be referenced by group.vcpp.compilers=&vcpp_x86:&vcpp_x64:&vcpp_arm64)
if self._is_group_referenced_elsewhere(group_name):
return # Already referenced by another group
compilers_line_idx = self.find_compilers_line()
if compilers_line_idx is None:
# No compilers line exists, create one
self.lines.insert(0, f"compilers=&{group_name}")
return
# Add to existing line
line = self.lines[compilers_line_idx]
if line.endswith("="):
# Empty compilers line, just append without colon
self.lines[compilers_line_idx] = f"{line}&{group_name}"
elif line.endswith(":"):
# Line ends with colon, just append
self.lines[compilers_line_idx] = f"{line}&{group_name}"
else:
# Add with colon separator
self.lines[compilers_line_idx] = f"{line}:&{group_name}"
def _is_group_referenced_elsewhere(self, group_name: str) -> bool:
"""Check if a group is referenced by any other group's compilers list."""
for line in self.lines:
# Look for group.*.compilers= lines that reference this group
if ".compilers=" in line and not line.startswith(f"group.{group_name}.compilers="):
# Extract the value part after =
if "=" in line:
value = line.split("=", 1)[1]
# Check if this group is referenced (with & prefix)
referenced_groups = []
for part in value.split(":"):
part = part.strip()
if part.startswith("&"):
referenced_groups.append(part[1:]) # Remove & prefix
if group_name in referenced_groups:
return True
return False
def group_exists(self, group_name: str) -> bool:
"""Check if a group already exists in the file."""
start_line, _ = self.find_group_section(group_name)
return start_line is not None
def compiler_exists(self, compiler_id: str) -> bool:
"""Check if a compiler already exists in the file."""
start_line, _ = self.find_compiler_section(compiler_id)
return start_line is not None
def find_insertion_point_for_group(self, group_name: str) -> int:
"""Find the best place to insert a new group section."""
# Find the end of both existing groups AND all compilers
last_group_end = 0
last_compiler_end = 0
# Find end of existing groups
for i, line in enumerate(self.lines):
if line.startswith("group."):
# Find end of this group
group_match = re.match(r"^group\.([^.]+)\.", line)
if group_match:
current_group = group_match.group(1)
_, end_line = self.find_group_section(current_group)
if end_line is not None:
last_group_end = max(last_group_end, end_line)
# Find end of all compilers
for i, line in enumerate(self.lines):
if line.startswith("compiler."):
# Find end of this compiler
compiler_match = re.match(r"^compiler\.([^.]+)\.", line)
if compiler_match:
current_compiler = compiler_match.group(1)
_, end_line = self.find_compiler_section(current_compiler)
if end_line is not None:
last_compiler_end = max(last_compiler_end, end_line)
# Insert after whichever comes last: groups or compilers
insertion_point = max(last_group_end, last_compiler_end)
# If neither groups nor compilers found, insert after compilers line
if insertion_point == 0:
compilers_line_idx = self.find_compilers_line()
if compilers_line_idx is not None:
# Insert after compilers line and any following blank lines
insertion_point = compilers_line_idx + 1
while insertion_point < len(self.lines) and self.lines[insertion_point].strip() == "":
insertion_point += 1
return insertion_point
else:
return 0
return insertion_point
def find_insertion_point_for_compiler(self, compiler_id: str, group_name: Optional[str] = None) -> int:
"""Find the best place to insert a new compiler section."""
# If we have a group, try to insert at the end of that group's compilers
if group_name:
group_start, group_end = self.find_group_section(group_name)
if group_start is not None:
# Look for compilers from this group after the group definition
last_compiler_end = group_end
# Find compilers that belong to this group
compilers_in_group = self.get_compilers_in_group(group_name)
for comp_id in compilers_in_group:
if comp_id != compiler_id: # Don't include ourselves
_, comp_end = self.find_compiler_section(comp_id)
if comp_end is not None:
last_compiler_end = max(last_compiler_end, comp_end)
return last_compiler_end
# Fallback: find the end of all compilers, but insert before libs/tools
last_compiler_end = 0
libs_tools_start = len(self.lines) # Default to end of file
# Find where libs/tools sections start
for i, line in enumerate(self.lines):
if (
line.startswith("libs=")
or line.startswith("tools=")
or (line.startswith("#") and ("####" in line or "Installed" in line))
):
libs_tools_start = i
break
# Find end of all compilers, but only those before libs/tools
for i, line in enumerate(self.lines):
if i >= libs_tools_start:
break
if line.startswith("compiler."):
# Find end of this compiler
compiler_match = re.match(r"^compiler\.([^.]+)\.", line)
if compiler_match:
current_compiler = compiler_match.group(1)
_, end_line = self.find_compiler_section(current_compiler)
if end_line is not None and end_line <= libs_tools_start:
last_compiler_end = max(last_compiler_end, end_line)
if last_compiler_end == 0:
# No compilers found, insert after groups but before libs/tools
group_insertion = self.find_insertion_point_for_group("dummy")
return min(group_insertion, libs_tools_start)
return min(last_compiler_end, libs_tools_start)
def get_compilers_in_group(self, group_name: str) -> List[str]:
"""Get list of compiler IDs in a group."""
group_start, group_end = self.find_group_section(group_name)
if group_start is None:
return []
# Look for group.{name}.compilers line
compilers_key = f"group.{group_name}.compilers"
for i in range(group_start, group_end):
line = self.lines[i]
if line.startswith(compilers_key + "="):
value = line.split("=", 1)[1]
# Parse compiler list (could be : separated or & prefixed)
compilers = []
for part in value.split(":"):
part = part.strip()
if part.startswith("&"):
part = part[1:] # Remove & prefix
if part:
compilers.append(part)
return compilers
return []
def add_compiler_to_group(self, group_name: str, compiler_id: str):
"""Add a compiler to a group's compilers list."""
group_start, group_end = self.find_group_section(group_name)
if group_start is None:
return # Group doesn't exist
# Find the group.{name}.compilers line
compilers_key = f"group.{group_name}.compilers"
for i in range(group_start, group_end):
line = self.lines[i]
if line.startswith(compilers_key + "="):
# Check if compiler is already in the list
existing_compilers = self.get_compilers_in_group(group_name)
if compiler_id in existing_compilers:
return # Already exists
# Add to the list
if line.endswith("="):
# Empty list
self.lines[i] = f"{line}{compiler_id}"
else:
# Add with colon separator
self.lines[i] = f"{line}:{compiler_id}"
return
def add_group_property(self, group_name: str, property_name: str, value: str):
"""Add a property to a group if it doesn't already exist."""
group_start, group_end = self.find_group_section(group_name)
if group_start is None:
return # Group doesn't exist
# Check if property already exists
prop_key = f"group.{group_name}.{property_name}"
for i in range(group_start, group_end):
line = self.lines[i]
if line.startswith(prop_key + "="):
return # Already exists
# Find a good place to insert (after the compilers line if it exists)
insertion_point = group_start + 1
compilers_key = f"group.{group_name}.compilers"
for i in range(group_start, group_end):
line = self.lines[i]
if line.startswith(compilers_key + "="):
insertion_point = i + 1
break
# Insert the new property
self.lines.insert(insertion_point, f"{prop_key}={value}")
def get_group_property(self, group_name: str, property_name: str) -> Optional[str]:
"""Get a property value from a group."""
group_start, group_end = self.find_group_section(group_name)
if group_start is None:
return None
# Check if property exists
prop_key = f"group.{group_name}.{property_name}"
for i in range(group_start, group_end):
line = self.lines[i]
if line.startswith(prop_key + "="):
return line.split("=", 1)[1]
return None
def add_compiler_property(self, compiler_id: str, property_name: str, value: str):
"""Add a property to a compiler if it doesn't already exist."""
compiler_start, compiler_end = self.find_compiler_section(compiler_id)
if compiler_start is None:
return # Compiler doesn't exist
# Check if property already exists
prop_key = f"compiler.{compiler_id}.{property_name}"
for i in range(compiler_start, compiler_end):
line = self.lines[i]
if line.startswith(prop_key + "="):
return # Already exists
# Insert at the end of the compiler section
insertion_point = compiler_end
# Try to insert in a logical order (exe, name, semver, compilerType, options, etc.)
desired_order = ["exe", "semver", "name", "compilerType", "options"]
if property_name in desired_order:
target_index = desired_order.index(property_name)
# Find where to insert based on order
for i in range(compiler_start, compiler_end):
line = self.lines[i]
if line.startswith(f"compiler.{compiler_id}."):
existing_prop = line.split(".", 2)[2].split("=")[0]
if existing_prop in desired_order:
existing_index = desired_order.index(existing_prop)
if existing_index > target_index:
insertion_point = i
break
else:
insertion_point = i + 1
# Insert the new property
self.lines.insert(insertion_point, f"{prop_key}={value}")
def create_group_section(self, group_name: str, compilers_list: Optional[List[str]] = None):
"""Create a new group section."""
if self.group_exists(group_name):
return # Already exists
insertion_point = self.find_insertion_point_for_group(group_name)
# Ensure proper spacing: blank line after compilers= and before group
compilers_line_idx = self.find_compilers_line()
if compilers_line_idx is not None and insertion_point == compilers_line_idx + 1:
# We're inserting right after compilers= line, add blank line first
self.lines.insert(insertion_point, "")
insertion_point += 1
elif (
insertion_point > 0 and insertion_point < len(self.lines) and self.lines[insertion_point - 1].strip() != ""
):
# Add empty line before group if previous line is not empty
self.lines.insert(insertion_point, "")
insertion_point += 1
# Create the group.{name}.compilers line
compilers_value = ":".join(compilers_list) if compilers_list else ""
self.lines.insert(insertion_point, f"group.{group_name}.compilers={compilers_value}")
def create_compiler_section(self, compiler: CompilerInfo):
"""Create a new compiler section."""
if self.compiler_exists(compiler.id):
return # Already exists
insertion_point = self.find_insertion_point_for_compiler(compiler.id, compiler.group)
# Ensure proper spacing: blank line after group section and before compiler
if compiler.group:
group_start, group_end = self.find_group_section(compiler.group)
if group_end is not None and insertion_point == group_end:
# We're inserting right after group section, add blank line first
self.lines.insert(insertion_point, "")
insertion_point += 1
# Add empty line before compiler if previous line is not empty
if insertion_point > 0 and insertion_point < len(self.lines) and self.lines[insertion_point - 1].strip() != "":
self.lines.insert(insertion_point, "")
insertion_point += 1
# Add compiler properties in order
props_to_add = []
# Normalize exe path for Windows (convert backslashes to forward slashes)
normalized_exe_path = compiler.exe.replace("\\", "/")
props_to_add.append(f"compiler.{compiler.id}.exe={normalized_exe_path}")
# Add semver if available, name if no semver or force_name is True
if compiler.semver:
props_to_add.append(f"compiler.{compiler.id}.semver={compiler.semver}")
if compiler.name and (not compiler.semver or compiler.force_name):
props_to_add.append(f"compiler.{compiler.id}.name={compiler.name}")
# Only add compilerType if the group doesn't already have the same one
if compiler.compiler_type:
group_compiler_type = None
if compiler.group:
group_compiler_type = self.get_group_property(compiler.group, "compilerType")
# Add compilerType only if group doesn't have it or has a different one
if not group_compiler_type or group_compiler_type != compiler.compiler_type:
props_to_add.append(f"compiler.{compiler.id}.compilerType={compiler.compiler_type}")
if compiler.options:
props_to_add.append(f"compiler.{compiler.id}.options={compiler.options}")
# Add Java-related properties for Java-based compilers
if compiler.java_home:
props_to_add.append(f"compiler.{compiler.id}.java_home={compiler.java_home}")
if compiler.runtime:
props_to_add.append(f"compiler.{compiler.id}.runtime={compiler.runtime}")
# Add execution wrapper for compilers that need it
if compiler.execution_wrapper:
props_to_add.append(f"compiler.{compiler.id}.executionWrapper={compiler.execution_wrapper}")
# Add MSVC-specific include and library paths
if compiler.include_path:
props_to_add.append(f"compiler.{compiler.id}.includePath={compiler.include_path}")
if compiler.lib_path:
props_to_add.append(f"compiler.{compiler.id}.libPath={compiler.lib_path}")
# Insert all properties
for prop in props_to_add:
self.lines.insert(insertion_point, prop)
insertion_point += 1
def ensure_libs_tools_sections(self):
"""Ensure libs= and tools= sections exist at the end if missing."""
has_libs = any(line.startswith("libs=") for line in self.lines)
has_tools = any(line.startswith("tools=") for line in self.lines)
if has_libs and has_tools:
# Check if there's proper spacing before libs section
self._ensure_proper_spacing_before_libs_tools()
return # Both exist
# Find insertion point (end of file, but before any existing libs/tools)
insertion_point = len(self.lines)
for i, line in enumerate(self.lines):
if line.startswith("libs=") or line.startswith("tools="):
insertion_point = i
break
# Add sections if missing
if not has_libs:
# Add libs section header
self.lines.insert(insertion_point, "")
self.lines.insert(insertion_point + 1, "#################################")
self.lines.insert(insertion_point + 2, "#################################")
self.lines.insert(insertion_point + 3, "# Installed libs")
self.lines.insert(insertion_point + 4, "libs=")
insertion_point += 5
if not has_tools:
# Add tools section header
self.lines.insert(insertion_point, "")
self.lines.insert(insertion_point + 1, "#################################")
self.lines.insert(insertion_point + 2, "#################################")
self.lines.insert(insertion_point + 3, "# Installed tools")
self.lines.insert(insertion_point + 4, "tools=")
def _ensure_proper_spacing_before_libs_tools(self):
"""Ensure there's proper spacing before libs/tools sections."""
# Find the start of libs/tools sections
libs_tools_start = None
for i, line in enumerate(self.lines):
if (
line.startswith("libs=")
or line.startswith("tools=")
or (line.startswith("#") and ("####" in line or "Installed" in line or "Libraries" in line))
):
libs_tools_start = i
break
if libs_tools_start is None:
return # No libs/tools sections found
# Check if there's an empty line before the libs/tools section
if libs_tools_start > 0 and self.lines[libs_tools_start - 1].strip() != "":
# No empty line before libs/tools, add one
self.lines.insert(libs_tools_start, "")
def ensure_proper_spacing_after_compiler(self, compiler_id: str):
"""Ensure proper spacing after a compiler section before libs/tools."""
compiler_start, compiler_end = self.find_compiler_section(compiler_id)
if compiler_start is None:
return
# Find if there are libs/tools sections after this compiler
libs_tools_start = None
for i in range(compiler_end, len(self.lines)):
line = self.lines[i]
if (
line.startswith("libs=")
or line.startswith("tools=")
or (line.startswith("#") and ("####" in line or "Installed" in line or "Libraries" in line))
):
libs_tools_start = i
break
if libs_tools_start is None:
return # No libs/tools sections after this compiler
# Check spacing between compiler end and libs/tools start
empty_lines_count = 0
for i in range(compiler_end, libs_tools_start):
if self.lines[i].strip() == "":
empty_lines_count += 1
else:
# Non-empty line found, reset count
empty_lines_count = 0
# Ensure exactly one empty line before libs/tools
if empty_lines_count == 0:
# No empty lines, add one
self.lines.insert(libs_tools_start, "")
elif empty_lines_count > 1:
# Too many empty lines, remove extras
lines_to_remove = empty_lines_count - 1
for _ in range(lines_to_remove):
for i in range(compiler_end, libs_tools_start):
if i < len(self.lines) and self.lines[i].strip() == "":
self.lines.pop(i)
break

View File

@@ -0,0 +1,286 @@
"""Shared utility functions for CE Properties Wizard."""
import re
import shutil
import subprocess
from pathlib import Path
from typing import Callable, List, Optional, Tuple
def find_ce_root_directory(search_targets: List[Tuple[str, Callable]], max_levels: int = 6) -> Optional[Path]:
"""Find CE root directory by looking for specific target paths.
Args:
search_targets: List of (relative_path, validation_function) tuples
max_levels: Maximum directory levels to traverse upward
Returns:
Path to CE root directory if found, None otherwise
"""
current_dir = Path(__file__).resolve().parent
for _ in range(max_levels):
for target_path, validator in search_targets:
target_dir = current_dir / target_path
if target_dir.exists() and validator(target_dir):
return current_dir
current_dir = current_dir.parent
return None
def find_ce_config_directory() -> Path:
"""Find the etc/config directory containing CE configuration files."""
def validate_config_dir(path: Path) -> bool:
return path.is_dir() and any(path.glob("*.defaults.properties"))
search_targets = [("etc/config", validate_config_dir)]
ce_root = find_ce_root_directory(search_targets)
if ce_root:
return ce_root / "etc" / "config"
# Fallback: check if we're already in the main CE directory
if Path("etc/config").exists() and Path("etc/config").is_dir():
config_dir = Path("etc/config").resolve()
if any(config_dir.glob("*.defaults.properties")):
return config_dir
raise FileNotFoundError("Could not find etc/config directory with CE configuration files")
def find_ce_lib_directory() -> Path:
"""Find the lib directory containing CE TypeScript files."""
def validate_lib_dir(path: Path) -> bool:
compilers_dir = path / "compilers"
return compilers_dir.exists() and compilers_dir.is_dir() and any(compilers_dir.glob("*.ts"))
search_targets = [("lib", validate_lib_dir)]
ce_root = find_ce_root_directory(search_targets)
if ce_root:
return ce_root / "lib"
# Fallback: assume we're in the main CE directory
lib_dir = Path("lib")
if validate_lib_dir(lib_dir):
return lib_dir.resolve()
raise FileNotFoundError("Could not find lib directory with TypeScript files")
def create_backup(file_path: Path) -> Path:
"""Create a backup of the file with .bak extension.
Args:
file_path: Path to the file to backup
Returns:
Path to the created backup file
"""
backup_path = file_path.with_suffix(".properties.bak")
if file_path.exists():
shutil.copy2(file_path, backup_path)
return backup_path
class SubprocessRunner:
"""Utility class for running subprocess commands with consistent error handling."""
@staticmethod
def run_with_timeout(
cmd: List[str], timeout: Optional[int] = 10, capture_output: bool = True, text: bool = True
) -> Optional[subprocess.CompletedProcess]:
"""Run a subprocess command with timeout and error handling.
Args:
cmd: Command and arguments to execute
timeout: Timeout in seconds (None for no timeout)
capture_output: Whether to capture stdout/stderr
text: Whether to return text output
Returns:
CompletedProcess result if successful, None if failed
"""
try:
# If timeout is None, run without timeout
if timeout is None:
result = subprocess.run(cmd, capture_output=capture_output, text=text)
else:
result = subprocess.run(cmd, capture_output=capture_output, text=text, timeout=timeout)
return result
except (subprocess.TimeoutExpired, subprocess.SubprocessError):
return None
class VersionExtractor:
"""Utility class for extracting version information from compiler output."""
# Regex patterns for different compiler types
PATTERNS = {
"gcc": [r"gcc.*?(\d+\.\d+\.\d+)", r"g\+\+.*?(\d+\.\d+\.\d+)"],
"clang": [r"clang version (\d+\.\d+\.\d+)"],
"intel": [r"(?:icc|icpc|icx|dpcpp).*?(\d+\.\d+\.\d+)", r"intel.*?compiler.*?(\d+\.\d+)"],
"intel_fortran": [r"(?:ifx|ifort)\s*\([^)]+\)\s*(\d+\.\d+\.\d+)", r"(?:ifx|ifort).*?(\d+\.\d+\.\d+)"],
"msvc": [r"compiler version (\d+\.\d+\.\d+)"],
"nvcc": [r"release (\d+\.\d+)"],
"rust": [r"rustc (\d+\.\d+\.\d+)"],
"go": [r"go\s*version\s+go(\d+\.\d+(?:\.\d+)?)", r"go(\d+\.\d+(?:\.\d+)?)"],
"tinygo": [r"(?:tinygo\s+)?version:?\s+(\d+\.\d+(?:\.\d+)?)"],
"python": [r"python (\d+\.\d+\.\d+)", r"pypy.*?(\d+\.\d+\.\d+)"],
"fpc": [r"Free Pascal Compiler version (\d+\.\d+\.\d+)", r"fpc.*?(\d+\.\d+\.\d+)"],
"z88dk": [r"z88dk.*?-\s*v([^-\s]+(?:-[^-\s]+)*)", r"v(\d+[^-\s]*(?:-[^-\s]*)*)"],
"kotlin": [r"kotlinc.*?(\d+\.\d+\.\d+)", r"kotlin.*?(\d+\.\d+\.\d+)"],
"zig": [r"zig (\d+\.\d+\.\d+)", r"zig.*?(\d+\.\d+\.\d+)"],
"dart": [r"Dart SDK version: (\d+\.\d+\.\d+)", r"dart.*?(\d+\.\d+\.\d+)"],
# Popular compiled languages
"dmd": [r"DMD.*?v(\d+\.\d+\.\d+)", r"dmd.*?(\d+\.\d+\.\d+)"],
"ldc": [r"LDC.*?(\d+\.\d+\.\d+)", r"ldc.*?(\d+\.\d+\.\d+)"],
"gdc": [r"gdc.*?(\d+\.\d+\.\d+)", r"GNU D compiler.*?(\d+\.\d+\.\d+)"],
"swiftc": [r"Swift version (\d+\.\d+(?:\.\d+)?)", r"swiftc.*?(\d+\.\d+(?:\.\d+)?)"],
"nim": [r"Nim Compiler Version (\d+\.\d+\.\d+)", r"nim.*?(\d+\.\d+\.\d+)"],
"crystal": [r"Crystal (\d+\.\d+\.\d+)", r"crystal.*?(\d+\.\d+\.\d+)"],
"v": [r"V (\d+\.\d+(?:\.\d+)?)", r"v.*?(\d+\.\d+(?:\.\d+)?)"],
# Functional languages
"ghc": [r"The Glorious Glasgow Haskell Compilation System, version (\d+\.\d+\.\d+)", r"ghc.*?(\d+\.\d+\.\d+)"],
"ocamlc": [r"OCaml version (\d+\.\d+\.\d+)", r"ocaml.*?(\d+\.\d+\.\d+)"],
"ocamlopt": [r"OCaml version (\d+\.\d+\.\d+)", r"ocaml.*?(\d+\.\d+\.\d+)"],
"scalac": [r"Scala compiler version (\d+\.\d+\.\d+)", r"scala.*?(\d+\.\d+\.\d+)"],
# .NET languages
"csharp": [r"Microsoft.*?C# Compiler version (\d+\.\d+\.\d+)", r"dotnet.*?(\d+\.\d+\.\d+)"],
"dotnet": [r"Microsoft.*?\.NET.*?(\d+\.\d+\.\d+)", r"dotnet.*?(\d+\.\d+\.\d+)"],
"fsharp": [r"F# Compiler.*?(\d+\.\d+\.\d+)", r"fsharpc.*?(\d+\.\d+\.\d+)"],
# Scripting/Dynamic languages
"ruby": [r"ruby (\d+\.\d+\.\d+)", r"ruby.*?(\d+\.\d+\.\d+)"],
"julia": [r"julia version (\d+\.\d+\.\d+)", r"julia.*?(\d+\.\d+\.\d+)"],
"elixir": [r"Elixir (\d+\.\d+\.\d+)", r"elixir.*?(\d+\.\d+\.\d+)"],
"erlc": [r"Erlang.*?(\d+(?:\.\d+)*)", r"erlc.*?(\d+(?:\.\d+)*)"],
# Assembly and low-level
"nasm": [r"NASM version (\d+\.\d+(?:\.\d+)?)", r"nasm.*?(\d+\.\d+(?:\.\d+)?)"],
"gas": [r"GNU assembler.*?(\d+\.\d+(?:\.\d+)?)", r"as.*?(\d+\.\d+(?:\.\d+)?)"],
"yasm": [r"yasm (\d+\.\d+\.\d+)", r"yasm.*?(\d+\.\d+\.\d+)"],
# Modern systems languages
"carbon": [r"Carbon.*?(\d+\.\d+(?:\.\d+)?)", r"carbon.*?(\d+\.\d+(?:\.\d+)?)"],
"mojo": [r"mojo (\d+\.\d+(?:\.\d+)?)", r"mojo.*?(\d+\.\d+(?:\.\d+)?)"],
"odin": [r"odin version (\d+\.\d+(?:\.\d+)?)", r"odin.*?(\d+\.\d+(?:\.\d+)?)"],
"gnatmake": [r"GNATMAKE.*?(\d+\.\d+(?:\.\d+)?)", r"gnat.*?(\d+\.\d+(?:\.\d+)?)"],
"gnucobol": [r"gnucobol.*?(\d+\.\d+(?:\.\d+)?)", r"cobol.*?(\d+\.\d+(?:\.\d+)?)"],
}
@classmethod
def extract_version(cls, compiler_type: str, output: str) -> Optional[str]:
"""Extract version string from compiler output.
Args:
compiler_type: Type of compiler (gcc, clang, etc.)
output: Raw output from compiler version command
Returns:
Extracted version string if found, None otherwise
"""
patterns = cls.PATTERNS.get(compiler_type, [])
for pattern in patterns:
match = re.search(pattern, output, re.IGNORECASE)
if match:
return match.group(1)
return None
@classmethod
def extract_semver(cls, version: Optional[str]) -> Optional[str]:
"""Extract semantic version from version string.
Args:
version: Version string to parse
Returns:
Semantic version (major.minor.patch) if found, None otherwise
"""
if not version:
return None
match = re.match(r"(\d+\.\d+(?:\.\d+)?)", version)
if match:
return match.group(1)
return None
class ArchitectureMapper:
"""Utility class for architecture and instruction set mapping."""
# Architecture mapping based on lib/instructionsets.ts
ARCH_MAPPINGS = {
"aarch64": "aarch64",
"arm64": "aarch64",
"arm": "arm32",
"avr": "avr",
"bpf": "ebpf",
"ez80": "ez80",
"kvx": "kvx",
"k1": "kvx",
"loongarch": "loongarch",
"m68k": "m68k",
"mips": "mips",
"mipsel": "mips",
"mips64": "mips",
"mips64el": "mips",
"nanomips": "mips",
"mrisc32": "mrisc32",
"msp430": "msp430",
"powerpc": "powerpc",
"ppc64": "powerpc",
"ppc": "powerpc",
"riscv64": "riscv64",
"rv64": "riscv64",
"riscv32": "riscv32",
"rv32": "riscv32",
"sh": "sh",
"sparc": "sparc",
"sparc64": "sparc",
"s390x": "s390x",
"vax": "vax",
"wasm32": "wasm32",
"wasm64": "wasm64",
"xtensa": "xtensa",
"z180": "z180",
"z80": "z80",
"x86_64": "amd64",
"x86-64": "amd64",
"amd64": "amd64",
"i386": "x86",
"i486": "x86",
"i586": "x86",
"i686": "x86",
}
@classmethod
def detect_instruction_set(cls, target: Optional[str], exe_path: str) -> str:
"""Detect instruction set from target platform or executable path.
Args:
target: Target platform string (e.g., from compiler -v output)
exe_path: Path to the compiler executable
Returns:
Instruction set name (defaults to "amd64" if not detected)
"""
if not target:
target = ""
target_lower = target.lower()
exe_lower = exe_path.lower()
# Check target first
for arch, instruction_set in cls.ARCH_MAPPINGS.items():
if arch in target_lower:
return instruction_set
# Check executable path as fallback
for arch, instruction_set in cls.ARCH_MAPPINGS.items():
if arch in exe_lower:
return instruction_set
# Default to amd64 if nothing detected
return "amd64"

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,39 @@
[tool.poetry]
name = "ce-properties-wizard"
version = "0.1.0"
description = "Interactive wizard for adding compilers to Compiler Explorer"
authors = ["Compiler Explorer Team"]
readme = "README.md"
packages = [{include = "ce_properties_wizard"}]
[tool.poetry.dependencies]
python = "^3.10"
click = "^8.1.7"
inquirer = "^3.1.3"
pydantic = "^2.5.0"
colorama = "^0.4.6"
[tool.poetry.group.dev.dependencies]
pytest = "^7.4.3"
black = "^23.11.0"
ruff = "^0.1.6"
pytype = "^2023.11.21"
[tool.poetry.scripts]
ce-props-wizard = "ce_properties_wizard.main:cli"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
[tool.black]
line-length = 120
target-version = ['py310']
[tool.ruff]
line-length = 120
select = ["E", "F", "I", "N", "W"]
[tool.pytype]
inputs = ['ce_properties_wizard']
python_version = '3.10'

View File

@@ -0,0 +1,100 @@
# CE Properties Wizard runner script for Windows PowerShell
param(
[Parameter(ValueFromRemainingArguments=$true)]
[string[]]$Arguments
)
# Get the directory where this script is located
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $ScriptDir
# Check if poetry is installed
if (-not (Get-Command poetry -ErrorAction SilentlyContinue)) {
Write-Host "Poetry is not installed. Installing Poetry..." -ForegroundColor Yellow
# Check if Python is available
$pythonCmd = $null
foreach ($cmd in @("python", "python3", "py")) {
if (Get-Command $cmd -ErrorAction SilentlyContinue) {
$pythonCmd = $cmd
break
}
}
if (-not $pythonCmd) {
Write-Host "Python is not installed. Please install Python first." -ForegroundColor Red
exit 1
}
try {
# Download and install Poetry
Write-Host "Downloading Poetry installer..." -ForegroundColor Green
$poetryInstaller = Invoke-RestMethod -Uri https://install.python-poetry.org
$poetryInstaller | & $pythonCmd -
# Update PATH for current session
$env:Path = "$env:APPDATA\Python\Scripts;$env:Path"
# Verify installation
if (-not (Get-Command poetry -ErrorAction SilentlyContinue)) {
Write-Host "Poetry installation failed. Please install manually from https://python-poetry.org/docs/#installation" -ForegroundColor Red
exit 1
}
Write-Host "Poetry installed successfully!" -ForegroundColor Green
} catch {
Write-Host "Failed to install Poetry automatically: $_" -ForegroundColor Red
Write-Host "Please install manually from https://python-poetry.org/docs/#installation" -ForegroundColor Yellow
exit 1
}
}
# Install dependencies if needed
if (-not (Test-Path ".venv")) {
Write-Host "Setting up virtual environment..." -ForegroundColor Green
# On Windows, use --only main to skip dev dependencies and avoid pytype build issues
poetry install --only main
Write-Host "Note: Development dependencies skipped on Windows (pytype doesn't build on Windows)" -ForegroundColor Yellow
}
# Check if we're running under Git Bash (which can cause issues with Poetry)
$isGitBash = $false
if ($env:SHELL -match "bash" -or $env:MSYSTEM) {
$isGitBash = $true
Write-Host "Warning: Git Bash detected. This may cause issues with Poetry." -ForegroundColor Yellow
# Find the virtual environment
$venvPython = Join-Path $ScriptDir ".venv\Scripts\python.exe"
if (-not (Test-Path $venvPython)) {
# Check Poetry's cache location
$poetryVenvs = "$env:LOCALAPPDATA\pypoetry\Cache\virtualenvs"
$venvDir = Get-ChildItem $poetryVenvs -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -like "ce-properties-wizard*" } | Select-Object -First 1
if ($venvDir) {
$venvPython = Join-Path $venvDir.FullName "Scripts\python.exe"
}
}
if (Test-Path $venvPython) {
Write-Host "Using Python at: $venvPython" -ForegroundColor Green
# Set UTF-8 encoding for Python to handle Unicode characters
$env:PYTHONIOENCODING = "utf-8"
if ($Arguments) {
& $venvPython -m ce_properties_wizard.main @Arguments
} else {
& $venvPython -m ce_properties_wizard.main
}
} else {
Write-Host "Could not find Python executable in virtual environment" -ForegroundColor Red
Write-Host "This might be due to Git Bash compatibility issues with Poetry on Windows" -ForegroundColor Yellow
Write-Host "Please run this script in a native PowerShell window instead" -ForegroundColor Yellow
exit 1
}
} else {
# Run the wizard with all arguments passed through
if ($Arguments) {
poetry run ce-props-wizard @Arguments
} else {
poetry run ce-props-wizard
}
}

View File

@@ -0,0 +1,78 @@
#!/bin/bash
# CE Properties Wizard runner script
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
# Check if poetry is installed
if ! command -v poetry &> /dev/null; then
echo "Poetry is not installed. Installing Poetry..."
# Check if Python is available
PYTHON_CMD=""
for cmd in python3 python py; do
if command -v $cmd &> /dev/null; then
PYTHON_CMD=$cmd
break
fi
done
if [ -z "$PYTHON_CMD" ]; then
echo "Python is not installed. Please install Python first."
exit 1
fi
# Install Poetry
echo "Downloading and installing Poetry..."
if curl -sSL https://install.python-poetry.org | $PYTHON_CMD -; then
# Add Poetry to PATH for current session
export PATH="$HOME/.local/bin:$PATH"
# Verify installation
if ! command -v poetry &> /dev/null; then
echo "Poetry installation failed. Please install manually from https://python-poetry.org/docs/#installation"
exit 1
fi
echo "Poetry installed successfully!"
else
echo "Failed to install Poetry automatically."
echo "Please install manually from https://python-poetry.org/docs/#installation"
exit 1
fi
fi
# Install dependencies if needed
if [ ! -d ".venv" ]; then
echo "Setting up virtual environment..."
poetry install
fi
# Check for --format parameter
if [[ "$1" == "--format" ]]; then
shift
if [[ "$1" == "--check" ]]; then
echo "Checking code formatting..."
poetry run black --check --diff .
echo "Checking code with ruff..."
poetry run ruff check .
echo "Running pytype..."
poetry run pytype .
echo "All formatting checks passed!"
else
echo "Formatting code with black..."
poetry run black .
echo "Formatting code with ruff..."
poetry run ruff check --fix .
echo "Running pytype..."
poetry run pytype .
echo "Code formatting complete!"
fi
exit 0
fi
# Run the wizard with all arguments passed through
poetry run ce-props-wizard "$@"

View File

@@ -32,6 +32,7 @@ from argparse import Namespace
parser = argparse.ArgumentParser(description='Checks for incorrect/suspicious properties.') parser = argparse.ArgumentParser(description='Checks for incorrect/suspicious properties.')
parser.add_argument ('--check-suspicious-in-default-prop', required=False, action="store_true") parser.add_argument ('--check-suspicious-in-default-prop', required=False, action="store_true")
parser.add_argument ('--config-dir', required=False, default="./etc/config") parser.add_argument ('--config-dir', required=False, default="./etc/config")
parser.add_argument ('--check-local', required=False, action="store_true")
PROP_RE = re.compile(r'([^# ]*)=(.*)#*') PROP_RE = re.compile(r'([^# ]*)=(.*)#*')
@@ -108,6 +109,7 @@ def process_file(file: str, args: Namespace):
listed_groups = set() listed_groups = set()
seen_groups = set() seen_groups = set()
no_compilers_list = set()
listed_compilers = set() listed_compilers = set()
seen_compilers_exe = set() seen_compilers_exe = set()
seen_compilers_id = set() seen_compilers_id = set()
@@ -134,7 +136,7 @@ def process_file(file: str, args: Namespace):
duplicated_compiler_references = set() duplicated_compiler_references = set()
duplicated_group_references = set() duplicated_group_references = set()
suspicious_check = args.check_suspicious_in_default_prop or not (file.endswith('.defaults.properties')) suspicious_check = args.check_suspicious_in_default_prop or (not file.endswith('.defaults.properties') and not file.endswith('.local.properties'))
suspicious_path = set() suspicious_path = set()
seen_typo_compilers = set() seen_typo_compilers = set()
@@ -235,7 +237,19 @@ def process_file(file: str, args: Namespace):
bad_tools_exe = listed_tools.symmetric_difference(seen_tools_exe) bad_tools_exe = listed_tools.symmetric_difference(seen_tools_exe)
bad_tools_id = listed_tools.symmetric_difference(seen_tools_id) bad_tools_id = listed_tools.symmetric_difference(seen_tools_id)
bad_default = default_compiler - listed_compilers bad_default = default_compiler - listed_compilers
if len(listed_compilers) == 0 and len(listed_groups) == 0:
allowed = ('execution.','compiler-explorer.', 'aws.', 'asm-docs.', 'builtin.', '.defaults.')
is_allowed_to_be_empty = False
for allow in allowed:
if allow in file:
is_allowed_to_be_empty = True
break
if not is_allowed_to_be_empty:
no_compilers_list.add(file)
return { return {
"no_compilers_list": no_compilers_list,
"not_a_valid_prop": not_a_valid_prop, "not_a_valid_prop": not_a_valid_prop,
"bad_compilers_exe": bad_compilers_exe - disabled, "bad_compilers_exe": bad_compilers_exe - disabled,
"bad_compilers_id": bad_compilers_ids - disabled, "bad_compilers_id": bad_compilers_ids - disabled,
@@ -256,11 +270,17 @@ def process_file(file: str, args: Namespace):
} }
def process_folder(folder: str, args): def process_folder(folder: str, args):
if not args.check_local:
return [(f, process_file(join(folder, f), args)) return [(f, process_file(join(folder, f), args))
for f in listdir(folder) for f in listdir(folder)
if isfile(join(folder, f)) if isfile(join(folder, f))
and not f.endswith('.local.properties') and not f.endswith('.local.properties')
and f.endswith('.properties')] and f.endswith('.properties')]
else:
return [(f, process_file(join(folder, f), args))
for f in listdir(folder)
if isfile(join(folder, f))
and f.endswith('.properties')]
def problems_found(file_result): def problems_found(file_result):
return any(len(file_result[r]) > 0 for r in file_result if r != "filename") return any(len(file_result[r]) > 0 for r in file_result if r != "filename")