mirrors/compiler-explorer
1
0
Fork 0
mirror of https://github.com/compiler-explorer/compiler-explorer.git synced 2025-12-27 07:04:04 -05:00
Code Issues Packages Projects Releases Wiki Activity

Add SPIR-V language with SPIRV-Tools compilers (#6947)

Browse Source 
for https://github.com/compiler-explorer/compiler-explorer/issues/2331

Hey, so I am currently trying to improve the SPIR-V ecosystem, in-charge
of the [SPIR-V Guide](https://github.com/KhronosGroup/SPIRV-Guide), and
part of the SPIR-V Working Group. We talked at our last Face-2-Face
gathering about getting more support for GPU tooling in something like
Compiler Explorer because you have all made this such an amazing tool!

I have enjoyed working in this codebase and happy to help keep improving
the SPIR-V as people find issues or want enhancements (I was already
about to spend time rebuilding Compiler Explorer, so happy to donate my
time to a single tool's effort)

(For those who haven't spent last few years staring at SPIR-V)

- SPIR-V is a an IR (very much like LLVM IR) but for GPUs
- SPIR-V can be used for Graphics and Compute
- Compute:
    - Can be tied to things like clang, llvm, mlir, OpenCL, etc
- One could take `spirv` as a language and compile it to something like
x86
      - (Future PRs for this realm of things)
- Graphics:
- Much harder to actually "compile" to any ISA because other graphics
pipeline information is missing that will effect the final ISA
- Microsoft has even now planned to move to [SPIR-V for DirectX
soon](https://devblogs.microsoft.com/directx/directx-adopting-spir-v/)
- Both:
- There are many tools that run on SPIR-V regardless (hence why they
share the same IR)
    - Being able to do tooling in compiler explorer would be AMAZING

This PR adds `SPIR-V` as a language and a new `SPIRV-Tools` compiler as
an initial compiler for it the language (since we already have the
SPIRV-Tools repo being pulled in on the infra side!)

I personally use `spirv-opt` and `spirv-val` the most and have been
testing this locally for a few days and feel it is ready to go. I plan
to slowly add more tools (ex. `spirv-fuzz`, `spirv-cross`,
`spirv-reflect`, etc)

Cheers!

(some screenshots of course)

![image](https://github.com/user-attachments/assets/419feab0-d030-4578-b32a-ef1cf95701da)

![image](https://github.com/user-attachments/assets/a00fa60f-7f05-4522-a9fc-1c43ecee1c42)
This commit is contained in:
Spencer Fricke
2024-10-28 22:24:50 -04:00
committed by GitHub
parent a13b52fd9e
commit 02603a6371
10 changed files with 319 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
.github/labeler.yml vendored
View File

@@ -340,6 +340,14 @@
- 'etc/config/spice.*.properties'
- 'static/modes/spice-mode.ts'
'lang-spirv':
- changed-files:
- any-glob-to-any-file:
- 'lib/compilers/spirv.ts'
- 'lib/compilers/spirv-tools.ts'
- 'etc/config/spirv.*.properties'
- 'static/modes/spirv-mode.ts'
'lang-swift':
- changed-files:
- any-glob-to-any-file:

21
etc/config/spirv.amazon.properties Normal file
View File

@@ -0,0 +1,21 @@
compilers=&spirv-opt:&spirv-val
defaultCompiler=spirv-opt
supportsBinary=false
supportsExecute=false
supportsAsmDocs=false
group.spirv-opt.groupName=Optimizer
group.spirv-opt.baseName=SPIRV-Tools Optimizer
group.spirv-opt.compilers=spirv-opt
group.spirv-opt.compilerType=spirv-tools
group.spirv-val.groupName=Validator
group.spirv-val.baseName=SPIRV-Tools Validator
group.spirv-val.compilers=spirv-val
group.spirv-val.compilerType=spirv-tools
assemblerPath=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-as
disassemblerPath=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-dis
compiler.spirv-opt.exe=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-opt
compiler.spirv-val.exe=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-val

21
etc/config/spirv.defaults.properties Normal file
View File

@@ -0,0 +1,21 @@
compilers=&spirv-opt:&spirv-val
defaultCompiler=spirv-opt
supportsBinary=false
supportsExecute=false
supportsAsmDocs=false
group.spirv-opt.groupName=Optimizer
group.spirv-opt.baseName=SPIRV-Tools Optimizer
group.spirv-opt.compilers=spirv-opt
group.spirv-opt.compilerType=spirv-tools
group.spirv-val.groupName=Validator
group.spirv-val.baseName=SPIRV-Tools Validator
group.spirv-val.compilers=spirv-val
group.spirv-val.compilerType=spirv-tools
assemblerPath=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-as
disassemblerPath=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-dis
compiler.spirv-opt.exe=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-opt
compiler.spirv-val.exe=/opt/compiler-explorer/SPIRV-Tools-master/build/tools/spirv-val

36
examples/spirv/default.spvasm Normal file
View File

@@ -0,0 +1,36 @@
; SPIR-V
; Version: 1.0
; Generator: Khronos Glslang Reference Front End; 7
; Bound: 19
; Schema: 0
OpCapability Shader
%1 = OpExtInstImport "GLSL.std.450"
OpMemoryModel Logical GLSL450
OpEntryPoint Fragment %main "main" %outFragColor %inColor
OpExecutionMode %main OriginUpperLeft
OpSource GLSL 450
OpName %main "main"
OpName %outFragColor "outFragColor"
OpName %inColor "inColor"
OpDecorate %outFragColor Location 0
OpDecorate %inColor Location 0
%void = OpTypeVoid
%3 = OpTypeFunction %void
%float = OpTypeFloat 32
%v4float = OpTypeVector %float 4
%_ptr_Output_v4float = OpTypePointer Output %v4float
%outFragColor = OpVariable %_ptr_Output_v4float Output
%v3float = OpTypeVector %float 3
%_ptr_Input_v3float = OpTypePointer Input %v3float
%inColor = OpVariable %_ptr_Input_v3float Input
%float_1 = OpConstant %float 1
%main = OpFunction %void None %3
%5 = OpLabel
%13 = OpLoad %v3float %inColor
%15 = OpCompositeExtract %float %13 0
%16 = OpCompositeExtract %float %13 1
%17 = OpCompositeExtract %float %13 2
%18 = OpCompositeConstruct %v4float %15 %16 %17 %float_1
OpStore %outFragColor %18
OpReturn
OpFunctionEnd

1
lib/compilers/_all.ts
View File

@@ -126,6 +126,7 @@ export {SolidityCompiler} from './solidity.js';
export {SolidityZKsyncCompiler} from './solidity-zksync.js';
export {SpiceCompiler} from './spice.js';
export {SPIRVCompiler} from './spirv.js';
export {SPIRVToolsCompiler} from './spirv-tools.js';
export {SwiftCompiler} from './swift.js';
export {TableGenCompiler} from './tablegen.js';
export {TenDRACompiler} from './tendra.js';

176
lib/compilers/spirv-tools.ts Normal file
View File

@@ -0,0 +1,176 @@
// Copyright (c) 2024, Compiler Explorer Authors
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
// POSSIBILITY OF SUCH DAMAGE.
import path from 'path';
import type {ExecutionOptionsWithEnv} from '../../types/compilation/compilation.interfaces.js';
import type {PreliminaryCompilerInfo} from '../../types/compiler.interfaces.js';
import type {ParseFiltersAndOutputOptions} from '../../types/features/filters.interfaces.js';
import {BaseCompiler} from '../base-compiler.js';
import {CompilationEnvironment} from '../compilation-env.js';
import {logger} from '../logger.js';
import {SPIRVAsmParser} from '../parsers/asm-parser-spirv.js';
import * as utils from '../utils.js';
// If you want to output SPIR-V, most likely you want SPIRVAsmParser
//
// SPIR-V is an IR that targets heterogeneous (like a GPU) and has a "compute" and "graphics" mode.
// When used for graphics, it inself is not enough info to "compile" down to the GPU's assembly because normally there
// is other graphics // pipeline information that is required (ex. is the depth test enable/disabled)
//
// SPIR-V has a lot of tooling around it to optimize, validate, fuzz, etc. This compiler is only used for tooling.
export class SPIRVToolsCompiler extends BaseCompiler {
protected assemblerPath: string;
protected disassemblerPath: string;
static get key() {
return 'spirv-tools';
}
constructor(compilerInfo: PreliminaryCompilerInfo, env: CompilationEnvironment) {
super(compilerInfo, env);
this.asm = new SPIRVAsmParser(this.compilerProps);
// spirv-as
this.assemblerPath = this.compilerProps<string>('assemblerPath');
// spirv-dis
this.disassemblerPath = this.compilerProps<string>('disassemblerPath');
}
override optionsForFilter(filters: ParseFiltersAndOutputOptions, outputFilename: string) {
const sourceDir = path.dirname(outputFilename);
const spvBinFilename = this.getPrimaryOutputFilename(sourceDir, this.outputFilebase);
return ['-o', spvBinFilename];
}
getPrimaryOutputFilename(dirPath: string, outputFilebase: string) {
return path.join(dirPath, `${outputFilebase}.spv`);
}
override getOutputFilename(dirPath: string, outputFilebase: string) {
return path.join(dirPath, `${outputFilebase}.spvasm`);
}
// Some tools (ex. spirv-opt) needs this as a single argument, so merge with "="
mergeSpirvTargetEnv(options: string[]) {
const index = options.indexOf('--target-env');
if (index !== -1) {
options.splice(index, 2, `--target-env=${options[index + 1]}`);
}
return options;
}
// Some tools (ex. spirv-val) needs this as a two argument, so unmerge with "="
unmergeSpirvTargetEnv(options: string[]) {
for (let i = 0; i < options.length; i++) {
if (options[i].indexOf('--target-env=') === 0) {
const parts = options[i].split('=');
options.splice(i, 1, parts[0], parts[1]);
break;
}
}
return options;
}
getSpirvTargetEnv(options: string[]) {
const index = options.indexOf('--target-env');
if (index !== -1) {
return [options[index], options[index + 1]];
}
for (const i in options) {
if (options[i].indexOf('--target-env=') === 0) {
return [options[i]];
}
}
return []; // no target found, use tool's default
}
// Most flows follow the same flow:
// 1. Assemble from spirv disassembly to a spirv binary
// 2. Run the tool (which will dump out a binary)
// a. Most tools let you set the input and out file to the same binary file
// 3. Disassemble back to disassembly
override async runCompiler(
compiler: string,
options: string[],
inputFilename: string,
execOptions: ExecutionOptionsWithEnv,
) {
const sourceDir = path.dirname(inputFilename);
const spvBinFilename = this.getPrimaryOutputFilename(sourceDir, this.outputFilebase);
if (!execOptions) {
execOptions = this.getDefaultExecOptions();
}
execOptions.customCwd = sourceDir;
let assemblerFlags = [inputFilename, '-o', spvBinFilename];
assemblerFlags = assemblerFlags.concat(this.getSpirvTargetEnv(options));
// Will fail if input SPIR-V is so bad assembler can't understand, so should let user know
let spvasmOutput = await this.exec(this.assemblerPath, assemblerFlags, execOptions);
let result = this.transformToCompilationResult(spvasmOutput, inputFilename);
if (spvasmOutput.code !== 0 || !(await utils.fileExists(spvBinFilename))) {
return result;
}
// needs to update options depending on the tool
const isValidator = compiler.endsWith('spirv-val');
if (isValidator) {
// there is no output file, so remove what we added in optionsForFilter
options = options.splice(2);
options = this.unmergeSpirvTargetEnv(options);
} else {
options = this.mergeSpirvTargetEnv(options);
}
// have tools input a binary and output it to same binary temp file
for (const i in options) {
if (options[i] === inputFilename) {
options[i] = spvBinFilename;
break;
}
}
const spvBin = await this.exec(compiler, options, execOptions);
result = this.transformToCompilationResult(spvBin, inputFilename);
if (spvBin.code !== 0 || !(await utils.fileExists(spvBinFilename)) || isValidator) {
return result;
}
const spvasmFilename = path.join(sourceDir, this.outputFilebase + '.spvasm');
const disassemblerFlags = [spvBinFilename, '-o', spvasmFilename];
// Will likely never fail
spvasmOutput = await this.exec(this.disassemblerPath, disassemblerFlags, execOptions);
if (spvasmOutput.code !== 0) {
logger.error('spirv-dis failed to disassemble binary', spvasmOutput);
}
result = this.transformToCompilationResult(spvasmOutput, spvBinFilename);
return result;
}
}

3
lib/compilers/spirv.ts
View File

@@ -1,4 +1,4 @@
// Copyright (c) 2018, 2021, Compiler Explorer Authors, Arm Ltd
// Copyright (c) 2018, 2021, 2024 Compiler Explorer Authors, Arm Ltd
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
@@ -38,6 +38,7 @@ import {logger} from '../logger.js';
import {SPIRVAsmParser} from '../parsers/asm-parser-spirv.js';
import * as utils from '../utils.js';
// If you want to output SPIR-V, most likely you want SPIRVAsmParser
export class SPIRVCompiler extends BaseCompiler {
protected translatorPath: string;
protected disassemblerPath: string;

11
lib/languages.ts
View File

@@ -726,6 +726,17 @@ const definitions: Record<LanguageKey, LanguageDefinition> = {
previewFilter: null,
monacoDisassembly: null,
},
spirv: {
name: 'SPIR-V',
monaco: 'spirv',
extensions: ['.spvasm'],
alias: [],
logoUrl: 'spirv.svg',
logoUrlDark: null,
formatter: null,
previewFilter: null,
monacoDisassembly: null,
},
swift: {
name: 'Swift',
monaco: 'swift',

1
types/languages.interfaces.ts
View File

@@ -85,6 +85,7 @@ export type LanguageKey =
| 'scala'
| 'solidity'
| 'spice'
| 'spirv'
| 'swift'
| 'tablegen'
| 'toit'

42
views/resources/logos/spirv.svg Normal file
View File

@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 18.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1 Tiny//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-tiny.dtd">
<svg version="1.1" baseProfile="tiny" id="SPIR" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
x="0px" y="0px" width="1000px" height="500px" viewBox="0 0 1000 500" xml:space="preserve">
<g>
<path fill="#004BA9" d="M676.1,362.2c-50.2,27.8-121.3,47.1-202.5,47.1c-146.2,0-264.8-58-264.8-153c0-101,118.5-156.7,264.8-156.7
c81.5,0,152.9,15.7,203.1,43C621.8,97,523.5,56.1,411.1,56.1C238.9,56,99.3,145.7,99.3,256.3c0,100.4,139.5,192.8,311.7,192.8
C523,449.2,621.1,408.8,676.1,362.2z"/>
</g>
<g>
<polygon fill="#004BA9" points="883.4,327 876,327 876,346.1 869.6,346.1 869.6,327 862.3,327 862.3,321.5 883.4,321.5 883.4,327
"/>
<polygon fill="#004BA9" points="886.2,321.5 895.7,321.5 899.9,337.9 900,337.9 904.2,321.5 913.7,321.5 913.7,346.1 907.7,346.1
907.7,327.4 907.6,327.4 902.4,346.1 897.5,346.1 892.3,327.4 892.3,327.4 892.3,346.1 886.2,346.1 886.2,321.5 "/>
</g>
<g>
<path fill="#004BA9" d="M313.9,282.7c0,6.4,1.2,11.8,3.5,16.3c2.3,4.4,5.4,8,9.2,10.8c3.8,2.8,8.3,4.8,13.5,6.1
c5.2,1.3,10.5,2,16,2c3.7,0,7.7-0.3,12-0.9c4.3-0.6,8.3-1.8,12-3.6c3.7-1.8,6.8-4.2,9.3-7.3c2.5-3.1,3.7-7.1,3.7-11.9
c0-5.2-1.6-9.3-4.9-12.5c-3.3-3.2-7.6-5.9-12.9-8c-5.3-2.1-11.4-4-18.1-5.6c-6.8-1.6-13.6-3.4-20.5-5.3c-7.1-1.8-14-4-20.8-6.5
c-6.8-2.6-12.8-5.9-18.1-10c-5.3-4.1-9.6-9.2-12.9-15.3c-3.3-6.1-4.9-13.5-4.9-22.3c0-9.8,2.1-18.3,6.3-25.4
c4.2-7.2,9.6-13.2,16.4-18c6.8-4.8,14.4-8.3,22.9-10.7c8.5-2.3,17.1-3.5,25.6-3.5c9.9,0,19.5,1.1,28.6,3.3
c9.1,2.2,17.3,5.8,24.4,10.8c7.1,5,12.7,11.3,16.9,19.1c4.2,7.7,6.3,17.1,6.3,28.1h-40.5c-0.4-5.7-1.6-10.4-3.6-14.1
c-2-3.7-4.8-6.7-8.1-8.8c-3.4-2.1-7.2-3.6-11.6-4.5c-4.4-0.9-9.1-1.3-14.3-1.3c-3.4,0-6.8,0.4-10.1,1.1c-3.4,0.7-6.4,2-9.2,3.7
c-2.8,1.8-5,4-6.8,6.7c-1.8,2.7-2.7,6-2.7,10.1c0,3.7,0.7,6.8,2.1,9.1c1.4,2.3,4.2,4.4,8.4,6.4c4.2,2,9.9,3.9,17.3,5.9
c7.4,2,17,4.4,28.9,7.5c3.6,0.7,8.5,2,14.8,3.9c6.3,1.9,12.6,4.8,18.8,8.9c6.2,4.1,11.6,9.5,16.1,16.4c4.5,6.8,6.8,15.6,6.8,26.2
c0,8.7-1.7,16.8-5.1,24.2c-3.4,7.5-8.4,13.9-15.1,19.3c-6.7,5.4-14.9,9.6-24.8,12.7c-9.9,3-21.3,4.5-34.2,4.5
c-10.5,0-20.7-1.3-30.5-3.9c-9.9-2.6-18.6-6.6-26.1-12.1c-7.6-5.5-13.5-12.5-18-21.1c-4.4-8.5-6.6-18.7-6.4-30.4H313.9z"/>
<path fill="#004BA9" d="M447.1,155.6h85.8c11.9,0,22,1.7,30.4,5.2c8.4,3.5,15.1,8,20.4,13.7c5.2,5.7,9.1,12.2,11.5,19.5
c2.4,7.3,3.6,14.8,3.6,22.7c0,7.6-1.2,15.1-3.6,22.5c-2.4,7.4-6.2,13.9-11.5,19.6c-5.2,5.7-12,10.3-20.4,13.7
c-8.3,3.5-18.5,5.2-30.4,5.2h-44v68.2h-41.8V155.6z M488.9,245.2h32.5c4.8,0,9.4-0.4,13.9-1.1c4.4-0.7,8.3-2.1,11.7-4.1
c3.4-2,6.1-4.9,8.1-8.7c2-3.7,3.1-8.6,3.1-14.7c0-6-1-10.9-3.1-14.7c-2-3.7-4.8-6.6-8.1-8.7c-3.4-2-7.3-3.4-11.7-4.1
c-4.4-0.7-9.1-1.1-13.9-1.1h-32.5V245.2z"/>
<path fill="#004BA9" d="M613.7,155.6h41.8v190.3h-41.8V155.6z"/>
<path fill="#004BA9" d="M681.1,155.6h102.6c8.5,0,16.2,1.4,23.1,4.1c6.8,2.8,12.7,6.5,17.6,11.3c4.9,4.8,8.6,10.3,11.2,16.7
c2.6,6.3,3.9,13.1,3.9,20.4c0,11.2-2.4,20.9-7.1,29c-4.7,8.2-12.4,14.4-23.1,18.7v0.5c5.2,1.4,9.4,3.6,12.8,6.5
c3.4,2.9,6.1,6.4,8.3,10.4c2.1,4,3.7,8.4,4.7,13.2c1,4.8,1.6,9.6,2,14.4c0.2,3,0.4,6.6,0.5,10.7c0.2,4.1,0.5,8.3,0.9,12.5
c0.4,4.3,1.2,8.3,2.1,12.1c1,3.8,2.4,7.1,4.4,9.7h-41.8c-2.3-6-3.7-13.2-4.3-21.6c-0.5-8.3-1.3-16.3-2.4-24
c-1.4-9.9-4.4-17.2-9.1-21.9c-4.6-4.6-12.2-6.9-22.7-6.9h-41.8v74.3h-41.8V155.6z M722.9,241.7h45.8c9.6,0,16.8-2.1,21.6-6.4
c4.8-4.3,7.2-11.2,7.2-20.8c0-9.2-2.4-15.9-7.2-20.1c-4.8-4.2-12-6.3-21.6-6.3h-45.8V241.7z"/>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.6 KiB