Files
compiler-explorer/test/asm-parser-tests.ts
Shawn Zhong 8befc91a79 Add Triton language and compiler (#7919)
Close #5530. Infra:
https://github.com/compiler-explorer/infra/pull/1711. Previous work by
@siboehm at #5531


## Summary

This pull request introduces support for the
[Triton](https://github.com/triton-lang/triton) language, a Python-based
DSL for writing highly efficient GPU kernels.


- [x] **New Language Support**: I've added comprehensive support for the
Triton programming language, allowing users to compile and inspect
Triton kernels within Compiler Explorer. (c.f.,
`lib/compilers/triton.ts`)
- [x] **Python Wrapper for Compilation**: A new Python wrapper script
(`triton_wrapper.py`) has been introduced to manage Triton compilation,
patching its behavior to dump compiled kernels and intermediate
representations without requiring actual execution, and consolidating
the output for Compiler Explorer.
- [x] **Device Assembly View**: Enables viewing of generated device
assembly code (e.g., PTX, AMDGCN) and various intermediate
representations (MLIR, LLVM IR) produced by the Triton compiler.
- [x] **MLIR Parsing**: New parsers (`asm-parser-mlir.ts` and
`mlir-pass-dump-parser.ts`) have been added to correctly interpret and
display MLIR assembly and optimization pass dumps, including source
location information.
- [x] **Multi-Version & Multi-Backend Support**: Painstakingly includes
all 8 versions (from 2.2.0 to 3.3.1) of Triton that supports Python
3.12. Supports both CUDA and HIP backend for Triton 3.

## Screenshots

Source and assembly:
<img width="1354" height="789" alt="image"
src="https://github.com/user-attachments/assets/c29650ff-2073-40e0-a9e6-ff8377094b5e"
/>



Device view for MLIR and LLVM IR: 
<img width="1402" height="670" alt="image"
src="https://github.com/user-attachments/assets/43dd5c68-ca78-41b1-9865-e97ffe3ef73c"
/>

Opt pipeline viewer:
<img width="1408" height="668" alt="image"
src="https://github.com/user-attachments/assets/429eef8c-aaac-4781-aafa-39ef0ffc7241"
/>

Diff of TTIR in Triton 3.3.1 vs 2.3.0:
<img width="1580" height="726" alt="image"
src="https://github.com/user-attachments/assets/a928c893-dd9a-4c3a-a048-14046e56a14c"
/>

CUDA & HIP:
<img width="1596" height="800" alt="image"
src="https://github.com/user-attachments/assets/c18800c3-cfad-4e5e-96de-ba92c9f236ea"
/>

## Implementation Details (and Notes for Reviewers)

- For Device Assembly View, I Implemented `MlirAsmParser` for parsing
MLIR assembly. Technically MLIR is not an assembly language, but there
is no better choice to make the source line map work w/ device view.
- I Implemented `MlirPassDumpParser` for processing MLIR optimization
pass dumps. I tried to subclass `LlvmPassDumpParser`, but they turn out
to be too different to worth doing it.
- `LlvmPassDumpParser` made some assumptions that do not hold true for
MLIR passed. Some effort is put to make sure that the passes are
properly diff-ed, since some passes can run multiple times and also
sometimes pass can be nested (i.e., some number of `before`s followed by
some number of `after`s)
- A lot of effort is put into `patch_triton` to make sure that the we
only compile the kernel without actually running it, and that needs to
work across all the versions we support.

## Steps to Run Locally

1. Clone https://github.com/ShawnZhong/compiler-explorer-infra.git
2. Install Triton to `/opt/compiler-explorer/triton`:
```sh
$ cd compiler-explorer-infra
$ ./bin/ce_install install triton
$ ls /opt/compiler-explorer/triton
# v2.2.0  v2.3.0  v2.3.1  v3.0.0  v3.1.0  v3.2.0  v3.3.0  v3.3.1
```
3. Clone https://github.com/ShawnZhong/compiler-explorer.git and
checkout branch `triton`
4. Run Compiler Explorer
```sh
make EXTRA_ARGS='--language triton' dev
```
5. Enjoy

---------

Co-authored-by: Matt Godbolt <matt@godbolt.org>
2025-07-30 10:15:28 -05:00

261 lines
11 KiB
TypeScript

// Copyright (c) 2024, Compiler Explorer Authors
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
// POSSIBILITY OF SUCH DAMAGE.
import {describe, expect, it} from 'vitest';
import {AsmParser} from '../lib/parsers/asm-parser.js';
import {MlirAsmParser} from '../lib/parsers/asm-parser-mlir.js';
import {PTXAsmParser} from '../lib/parsers/asm-parser-ptx.js';
describe('AsmParser tests', () => {
const parser = new AsmParser();
it('should identify generic opcodes', () => {
expect(parser.hasOpcode(' mov r0, #1')).toBe(true);
expect(parser.hasOpcode(' ROL A')).toBe(true);
});
it('should not identify non-opcodes as opcodes', () => {
expect(parser.hasOpcode(' ;mov r0, #1')).toBe(false);
expect(parser.hasOpcode('')).toBe(false);
expect(parser.hasOpcode('# moose')).toBe(false);
});
it('should identify llvm opcodes', () => {
expect(parser.hasOpcode(' %i1 = phi i32 [ %i2, %.preheader ], [ 0, %bb ]')).toBe(true);
});
});
describe('AsmParser comment filtering', () => {
const parser = new AsmParser();
it('should keep label lines starting with @ when filtering comments', () => {
const input = '@cube@4:\n ret';
const result = parser.processAsm(input, {commentOnly: true});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('@cube@4:');
expect(lines[1]).toBe(' ret');
});
});
describe('PTXAsmParser tests', () => {
const parser = new PTXAsmParser();
describe('Nested brace indentation', () => {
it('should indent content inside callseq blocks', () => {
const input = `{ // callseq 0, 0
.param .b64 param0;
st.param.b64 [param0+0], %rd2;
}`;
const result = parser.processAsm(input, {});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('{ // callseq 0, 0');
expect(lines[1]).toBe('\t.param .b64 param0;');
expect(lines[2]).toBe('\tst.param.b64 [param0+0], %rd2;');
expect(lines[3]).toBe('}');
});
it('should properly indent function calls inside nested braces', () => {
const input = `{ // callseq 0, 0
call.uni (retval0),
vprintf,
(
param0,
param1
);
}`;
const result = parser.processAsm(input, {});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('{ // callseq 0, 0');
expect(lines[1]).toBe('\tcall.uni (retval0),');
expect(lines[2]).toBe('\tvprintf,');
expect(lines[3]).toBe('\t(');
expect(lines[4]).toBe('\tparam0,');
expect(lines[5]).toBe('\tparam1');
expect(lines[6]).toBe('\t);');
expect(lines[7]).toBe('}');
});
it('should have proper indentation logic', () => {
const input = `.visible .entry _Z6kernelv()
{
mov.u64 %rd1, $str;
{ // callseq 0, 0
call.uni (retval0),
vprintf,
(
param0,
param1
);
}
}`;
const result = parser.processAsm(input, {});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('.visible .entry _Z6kernelv()');
expect(lines[1]).toBe('{');
expect(lines[2]).toBe('\tmov.u64 %rd1, $str;');
expect(lines[3]).toBe('\t{ // callseq 0, 0');
expect(lines[4]).toBe('\t\tcall.uni (retval0),');
expect(lines[5]).toBe('\t\tvprintf,');
expect(lines[6]).toBe('\t\t(');
expect(lines[7]).toBe('\t\tparam0,');
expect(lines[8]).toBe('\t\tparam1');
expect(lines[9]).toBe('\t\t);');
expect(lines[10]).toBe('\t}');
expect(lines[11]).toBe('}');
});
});
describe('Label handling', () => {
it('should never indent labels', () => {
const input = `{ // callseq 0, 0
$L__BB0_3:
mov.u64 %rd1, $str;
}`;
const result = parser.processAsm(input, {});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('{ // callseq 0, 0');
expect(lines[1]).toBe('$L__BB0_3:');
expect(lines[2]).toBe('\tmov.u64 %rd1, $str;');
expect(lines[3]).toBe('}');
});
});
describe('Trim filter', () => {
it('should convert tabs to spaces when trim filter is enabled', () => {
const input = `\t\t\tcall.uni (retval0),
\t\t\tvprintf,
\t\t\t(
\t\t\tparam0,
\t\t\tparam1
\t\t\t);`;
const result = parser.processAsm(input, {trim: true});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe(' call.uni (retval0),');
expect(lines[1]).toBe(' vprintf,');
expect(lines[2]).toBe(' (');
expect(lines[3]).toBe(' param0,');
expect(lines[4]).toBe(' param1');
expect(lines[5]).toBe(' );');
});
it('should preserve structure but compress whitespace with trim filter', () => {
const input = `{ // callseq 0, 0
\t\t\tcall.uni (retval0),
\t\t\tvprintf,
}`;
const result = parser.processAsm(input, {trim: true});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('{ // callseq 0, 0');
expect(lines[1]).toBe(' call.uni (retval0),');
expect(lines[2]).toBe(' vprintf,');
expect(lines[3]).toBe('}');
});
it('should preserve nested indentation structure with trim filter', () => {
const input = `.visible .entry _Z6kernelv()
{
mov.u64 %rd1, $str;
{ // callseq 0, 0
call.uni (retval0),
vprintf,
}
}`;
const result = parser.processAsm(input, {trim: true});
const lines = result.asm.map(line => line.text);
expect(lines[0]).toBe('.visible .entry _Z6kernelv()');
expect(lines[1]).toBe('{');
expect(lines[2]).toBe(' mov.u64 %rd1, $str;');
expect(lines[3]).toBe(' { // callseq 0, 0');
expect(lines[4]).toBe(' call.uni (retval0),');
expect(lines[5]).toBe(' vprintf,');
expect(lines[6]).toBe(' }');
expect(lines[7]).toBe('}');
});
});
});
describe('MlirAsmParser tests', () => {
const parser = new MlirAsmParser();
describe('Location handling', () => {
it('should process MLIR with location references', () => {
const input = `
#loc = loc("<source>":7:0)
module {
tt.func public @add_kernel(%arg0: !tt.ptr<f32> {tt.divisibility = 16 : i32} loc("<source>":7:0), %arg1: !tt.ptr<f32> {tt.divisibility = 16 : i32} loc("<source>":7:0), %arg2: !tt.ptr<f32> {tt.divisibility = 16 : i32} loc("<source>":7:0), %arg3: i32 {tt.divisibility = 16 : i32} loc("<source>":7:0)) attributes {noinline = false} {
%c1024_i32 = arith.constant 1024 : i32 loc(#loc1)
%0 = tt.get_program_id x : i32 loc(#loc2)
} loc(#loc)
} loc(#loc)
#loc1 = loc(unknown)
#loc2 = loc("<source>":14:24)`;
const result = parser.processAsm(input, {});
// Verify location definitions are removed
expect(result.asm.find(line => line.text.includes('#loc = loc'))).toBeUndefined();
expect(result.asm.find(line => line.text.includes('#loc1 = loc'))).toBeUndefined();
expect(result.asm.find(line => line.text.includes('#loc2 = loc'))).toBeUndefined();
// Verify location references are removed from displayed text
expect(result.asm.find(line => line.text.includes('loc(#loc)'))).toBeUndefined();
expect(result.asm.find(line => line.text.includes('loc(#loc1)'))).toBeUndefined();
expect(result.asm.find(line => line.text.includes('loc(#loc2)'))).toBeUndefined();
// Verify inline locations are removed
expect(result.asm.find(line => line.text.includes('loc("<source>"'))).toBeUndefined();
// Verify source information is correctly associated
const programIdLine = result.asm.find(line => line.text.includes('tt.get_program_id'));
expect(programIdLine).toBeDefined();
expect(programIdLine?.source).toBeDefined();
expect(programIdLine?.source?.file).toBe('<source>');
expect(programIdLine?.source?.line).toBe(14);
expect(programIdLine?.source?.column).toBe(24);
// Verify unknown locations are not associated with source information
const constantLine = result.asm.find(line => line.text.includes('arith.constant'));
expect(constantLine).toBeDefined();
expect(constantLine?.source).toBeNull();
// Verify the structure is preserved
const moduleStartLine = result.asm.find(line => line.text.includes('module {'));
expect(moduleStartLine).toBeDefined();
const funcLine = result.asm.find(line => line.text.includes('tt.func public @add_kernel('));
expect(funcLine).toBeDefined();
expect(funcLine?.text?.includes('%arg0: !tt.ptr<f32> {tt.divisibility = 16 : i32}, ')).toBe(true);
expect(funcLine?.text?.includes('%arg1: !tt.ptr<f32> {tt.divisibility = 16 : i32}, ')).toBe(true);
expect(funcLine?.text?.includes('%arg2: !tt.ptr<f32> {tt.divisibility = 16 : i32}, ')).toBe(true);
expect(funcLine?.text?.includes('%arg3: i32 {tt.divisibility = 16 : i32})')).toBe(true);
const constLine = result.asm.find(line => line.text.includes('arith.constant 1024'));
expect(constLine).toBeDefined();
});
});
});