petrbalvin/gasm-sdk
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity

docs: bring the document set into the standard shape
Test / test (push) Successful in 2m28s
Details

Browse Source 
Assisted-by: GLM 5.3 Flash
This commit is contained in:
petrbalvin
2026-09-17 20:33:18 +02:00
parent 03d6d4da54
commit c834d98210
7 changed files with 700 additions and 477 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+98 -13
View File
@@ -4,7 +4,9 @@ How gasm-devkit is put together and why.
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
## Design goals
## Overview
Three design goals shape everything below.
1. **A real AST, not a grammar hack.** The linter, analyser, assembler and
language server all need to *reason* about assembly, not just colour it.
@@ -20,10 +22,10 @@ Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrb
through two vendor-neutral interfaces: a CLI and an LSP server. No editor
owns the toolkit; the toolkit is offered to editors on standard terms.
## Pipeline
The components, and how data moves between them:
```mermaid
graph TD
flowchart TD
SRC["source .s"] --> LEX["lexer<br/>token stream"]
LEX --> PAR["parser<br/>AST + diagnostics"]
LEX --> FMT["format<br/>re-space tokens"]
@@ -42,9 +44,34 @@ graph TD
The lexer is the shared foundation: the parser builds the AST from it, the
formatter re-spaces its tokens directly, and the language server uses it for
semantic highlighting.
semantic highlighting. The phases follow a dependency chain: Phase 1 (static
analysis) builds only on the AST, Phase 2 (the standalone assembler) emits
object code, and Phases 3 (dynamic analysis) and 4 (the debugger) both consume
the execution substrate that the assembler provides.
## Components
## Packages
| Package | Responsibility |
|---|---|
| `token` | token kinds and positions |
| `lexer` | hand-written scanner; permissive, and it never panics |
| `ast` | the typed syntax tree: declarations, lines, operands |
| `parser` | line-oriented parser producing the AST and its diagnostics |
| `arch` | register and instruction tables for the four architectures |
| `lint` | static checks over the AST |
| `format` | canonical formatter over the token stream |
| `lsp` | the language server |
| `asm` | standalone assembler: encoders, image layout, object emitters |
| `verify` | JIT execution, ABI checks, differential fuzzing |
| `debug` | interactive ptrace debugger |
| `cmd/gasm` | the CLI |
| `_gen` | rebuilds the `arch` tables from the Go toolchain source |
The boundaries matter as much as the responsibilities: `ast` records syntax
only, and whether a name is a register or a label is left to `arch`, so the
parser stays architecture-agnostic. `asm` and `verify` are the only packages
that touch machine code and executable memory, and `cmd/gasm` owns no logic
beyond flags and output.
### `token` and `lexer`
@@ -206,7 +233,7 @@ The standalone assembler (Phase 2). Its core is an amd64 instruction encoder:
a REX/ModR-M/SIB/displacement/immediate engine plus the scalar instruction set,
with the Plan 9 operand order (source first) mapped onto the x86 encoding.
Every encoding is validated by decoding it again with `golang.org/x/arch`, the
one module dependency, used in tests only and never linked into the binary.
one module dependency, which also backs the `gasm dis` listings.
A **RISC-V encoder** (Phase 5, RV64IMAFDC + RVC compression) encodes the full
integer, atomic, float/double, FMA and CSR instruction sets with the MOV
@@ -383,8 +410,8 @@ the Go ABI fixes across calls (amd64 `BP`/`R14`, arm64 `R29`/`R28`, riscv64
raw return trampoline `leaveJITCheckedRaw` verifies them, restoring the
saved registers before Go code resumes. riscv64 is validated end to
end under qemu-user emulation; arm64 shares the same stack convention and
fix; loong64 stays ground-truth-only until hardware validation (see
docs/DECISIONS.md). `gasm verify` runs the JIT checks when the host
fix; loong64 stays ground-truth-only until hardware validation.
`gasm verify` runs the JIT checks when the host
matches the kernel's architecture and the toolchain comparisons
elsewhere.
@@ -436,14 +463,72 @@ watchdog is armed before the ptrace attach, so a sandboxed debuggee cannot
block it), and `--cover` runs to completion with a breakpoint on every
label and reports which blocks executed.
## Extension points
### Extending the toolkit
- **New architecture:** add an entry to the generator in `_gen`, run
`just gen`, and add a `buildXXX()` register file plus a case in `ForArch`.
- **New lint rule:** add a function in `lint` and a rule-code constant.
- **New LSP feature:** add a method case in `dispatch` and a handler.
The phases follow a dependency chain. Phase 1 (static analysis) builds only on
the AST; Phase 2 (the standalone assembler) emits object code; Phases 3
(dynamic analysis) and 4 (the debugger) both consume the execution substrate
that the assembler provides.
## Data flow
The main operation, assembling one file:
```mermaid
sequenceDiagram
participant User
participant CLI as gasm CLI
participant Parser as parser
participant Asm as asm
participant Go as go toolchain
User->>CLI: gasm asm --format goobj -p pkg -o k.o k_amd64.s
CLI->>Parser: Parse(path, src)
Parser-->>CLI: AST, diagnostics
CLI->>Asm: AssembleFile(AST)
Asm->>Asm: encode operands, settle label offsets, lay out data
Asm-->>CLI: Image, code and data and relocations
CLI->>Asm: GOObject(pkg, path)
Asm->>Go: go list -json -export, externals only
Go-->>Asm: package and symbol indices
Asm-->>CLI: Go object bytes
CLI-->>User: wrote N bytes to k.o
```
Errors are produced where the parse or the encoding fails and become values at
the CLI boundary: the parser returns a diagnostic list and never aborts a file,
`AssembleFile` returns an error, and `cmd/gasm` prints what it has to stderr
and returns a non-zero exit code. The formatter and the linter take the same
AST by a different route: `gasm fmt` re-spaces the token stream and `gasm lint`
walks the parsed file, so neither depends on an encoding.
## State and lifetime
- The analysis packages (`lexer`, `parser`, `format`, `lint`, `arch`) hold only
read-only lookup tables and no mutable state: every call allocates its own
tokens and AST, and any number of goroutines may read the `arch` tables.
- A `verify.Kernel` owns one executable mapping, which `Close` releases. The
JIT trampolines keep the Go stack pointer and the checked-call sentinels in
package globals, so a call is a process-wide, one-at-a-time operation. The
`gasm verify` sweeps therefore run each function in a child process, which
contains a crash and keeps the globals unshared.
- `lsp.Server` is long-lived: it runs a single read and dispatch loop over the
stream and touches its document store only from that loop, so one server
serves one connection.
- A `debug.Session` owns a traced child process and pins its goroutine to the
forking OS thread, because ptrace requests must stay on that thread.
## Dependencies
- **`golang.org/x/arch`** (v0.30.0) is the one module dependency: it is the
disassembler backend (`gasm dis` and the debugger's listings) and the source
of the register metadata the encoder consults (`asm/reg.go`, `asm/vex.go`).
The tests additionally decode through it to validate the encodings.
- **The Go toolchain**, as an oracle and never as a library: `go tool asm`
supplies the object preamble and the ground truth for `gasm verify
--ground-truth`, `go list -json -export` locates the archives of the packages
a GOOBJ object references, and `_gen` parses
`$GOROOT/src/cmd/internal/obj/<arch>/anames.go` to rebuild the tables.
- **Linux process interfaces** for the dynamic work: `mmap` and `mprotect` for
the JIT mapping, ptrace with `/proc/pid/mem` for the debugger. That is why
`verify` runs a JIT check only when the host architecture matches the
kernel's, and why `debug` is Linux-only.
docs/CLI.md
+387 -162
View File
@@ -1,215 +1,440 @@
# CLI Reference
# Command line
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
The reference below is taken from the program's own `--help`. If the two disagree, the
program is right and this file is a defect.
`gasm` is a single binary with subcommands. Run `gasm --help` for an
overview, or `gasm <command> -h` for a command's usage and flags.
## Synopsis
## Global Flags
```sh
gasm [global flags] <command> [command flags] [arguments]
```
| Flag | Description |
|------|-------------|
| `-h`, `--help` | Show help |
| `-V`, `--version` | Print the version |
## Commands
## `gasm tokens <file>`
| Command | Purpose |
|---|---|
| `tokens` | print the lexical token stream |
| `parse` | parse a file and report syntax errors |
| `fmt` | canonicalise the formatting of `.s` files |
| `lint` | run the static checks |
| `asm` | assemble `.s` files to machine code |
| `dis` | disassemble machine code or an assembled file |
| `verify` | JIT-assemble and run the dynamic checks |
| `debug` | interactive source-level debugger |
| `diff` | compare the machine code of two `.s` files |
| `profile` | show the basic-block structure of the functions |
| `audit-instructions` | diff the encoder against the toolchain's name table |
| `scaffold` | generate a differential test skeleton for a kernel |
| `lsp` | run the language server over stdio |
| `version` | print the version |
Print the lexical token stream of FILE: position, token kind, and text,
one token per line. FILE may be `-` to read standard input.
## tokens
## `gasm parse <file>`
```text
Usage: gasm tokens <file>
```
Parse FILE and report syntax errors on stderr. On success, prints how
many declarations and TEXT functions the file contains.
Print the lexical token stream of FILE: position, token kind and text, one
token per line. FILE may be `-` to read standard input.
## `gasm fmt [-w|-l|-d] [path...]`
```sh
gasm tokens hello_amd64.s
```
Canonicalise the formatting of Plan 9 assembly sources: indentation,
operand spacing, per-function mnemonic alignment, and blank-line layout.
```text
1:1 # "#"
1:2 IDENT "include"
1:10 STRING "\"textflag.h\""
```
| Flag | Description |
|------|-------------|
| `-w` | Write result to the source file (default: print to stdout) |
| `-l` | List files whose formatting differs, one per line; write nothing |
| `-d` | Print a unified diff of the canonical formatting instead |
## parse
With no arguments, or with a directory argument, every `.s` file below
it is reformatted in place and the names of changed files are listed
(`go fmt` style). `.` and `_` directories are skipped.
```text
Usage: gasm parse <file>
```
## `gasm lint <file...>`
Parse FILE and report syntax errors on stderr. On success, print how many
declarations and TEXT functions the file contains.
Run static checks and print diagnostics as
`file:line:col: severity: message [code]`. Exit status is non-zero when
an error-severity diagnostic is found.
```sh
gasm parse hello_amd64.s
```
| Flag | Description |
|------|-------------|
| `-disable` | Comma-separated rule codes to disable |
```text
hello_amd64.s: OK, 2 declarations, 1 functions
```
## fmt
```text
Usage: gasm fmt [-w|-l|-d] [path...]
```
| Flag | Default | Effect |
|---|---|---|
| `-w` | off | write the result back to the source file |
| `-l` | off | list the files whose formatting differs; write nothing |
| `-d` | off | print a unified diff of the canonical formatting instead |
`-l` and `-d` are mutually exclusive. With no arguments, or with a directory
argument, every `.s` file below it is reformatted in place and the names of the
changed files are listed, the way `go fmt` does; `.` and `_` directories are
skipped. Explicit file arguments print to stdout unless `-w` is given.
```sh
gasm fmt -l kernel_amd64.s
```
Empty output means every file is formatted, which is the shape a CI check
wants; `-d` shows what would change:
```sh
gasm fmt -d ugly_amd64.s
```
```text
--- ugly_amd64.s
+++ ugly_amd64.s
@@ -2,8 +2,8 @@
// func add(a, b int) int
TEXT ·add(SB), NOSPLIT, $0-24
- MOVQ a+0(FP), AX
- ADDQ b+8(FP), AX
+ MOVQ a+0(FP), AX
+ ADDQ b+8(FP), AX
```
## lint
```text
Usage: gasm lint <file...>
```
| Flag | Default | Effect |
|---|---|---|
| `-disable` | empty | comma-separated rule codes to disable |
Diagnostics are printed as `file:line:col: severity: message [code]`. The exit
status is non-zero when an error-severity diagnostic is found; warnings (the
register-clobber audit, for example) do not affect it.
Rules: `unknown-instruction`, `operand-count`, `undefined-label`,
`duplicate-label`, `missing-ret`, `missing-textflag-include`,
`abi-argsize`, `unreachable-code`, `register-clobber`,
`funcdata-pcdata`, `unused-label`, `invalid-textflag`,
`stack-imbalance`, `register-width-mismatch`, `abi0-register-args`,
`nonportable-register-name` and `unencodable-instruction`.
`nonportable-register-name`, `unencodable-instruction` and
`reserved-register-write`.
## `gasm asm [--format raw|elf|goobj] [-p pkg] [-o out] <file>`
```sh
gasm lint kernel_amd64.s
```
Assemble FILE to machine code (amd64, arm64, riscv64, loong64).
## asm
| Flag | Description |
|------|-------------|
| `--format` | Output format: `raw` (default), `elf`, `goobj` |
| `-p` | Package path (required for `--format goobj`) |
| `-o` | Write output to file (default: hex dump to stdout) |
```text
Usage: gasm asm [--format raw|elf|goobj] [-p pkg] [-o out] <file>
```
## `gasm dis [-a arch] <file>`
| Flag | Default | Effect |
|---|---|---|
| `-format` | `raw` | output format: `raw` (concatenated image), `elf` or `goobj` (Go object) |
| `-p` | empty | package path for `--format goobj`, qualifying the exported symbols |
| `-o` | empty | write the output to this file instead of a hex dump on stdout |
Disassemble machine code to instruction text (via `golang.org/x/arch`).
Supported architectures: amd64 (VEX/AVX2 and EVEX/AVX-512 included), arm64,
riscv64 (RV64IMAFDC and RVC) and loong64, selected from the file's `_arch.s`
suffix. `raw` concatenates the functions and the data section into one
self-consistent image; `elf` emits a relocatable object that links with the
system toolchain; `goobj` emits the Go toolchain's own object format, which
`cmd/link` consumes directly.
With a `.s` file, the file is assembled first and the listing follows the
real layout: one block per `TEXT` function, local labels printed at their
offsets. The architecture comes from the file name suffix, or from `-a`.
With any other file, or `-` for standard input, the bytes are
disassembled linearly and `-a` selects the architecture (amd64, arm64,
riscv64 or loong64).
```sh
gasm asm hello_amd64.s
```
| Flag | Description |
|------|-------------|
| `-a` | Architecture for raw input without a `_arch.s` name |
```text
add: 16 bytes
0000: 48 8b 44 24 08 48 03 44 24 10 48 89 44 24 18 c3
```
## `gasm verify [flags] <file.s>`
## dis
Assemble FILE, map it into executable memory, and run dynamic checks.
```text
Usage: gasm dis [-a arch] <file>
```
| Flag | Description |
|------|-------------|
| `--ground-truth` | Compare machine code byte-for-byte against `go tool asm` |
| `--fuzz` | Differential fuzz: JIT both gasm and go-tool-asm, compare outputs |
| `-n` | Fuzz iterations per function (default: 1000) |
| `--abi` | Run ABI-checking calls (sentinel registers + red zone) |
| `--abi-n` | Number of ABI check iterations with varied inputs (default: 100) |
| `--profile` | List basic-block structure per function |
| `--smoke` | Call each NOSPLIT function with zeroed args |
| `--call <func>` | Invoke a single function with `--buf` instead of the sweeps |
| `--buf <spec>` | Buffer spec for `--call`: `name:size:pattern[,name:size:pattern]` |
| `--args <spec>` | Scalar args for `--call`: `name=value[,name=value]` (decimal or `0x` hex) |
| `--repeat <n>` | Number of times to repeat a `--call` invocation (default: 1) |
| `--save-corpus <dir>` | With `--fuzz`: write each failing input to DIR as replayable JSON |
| `--replay <dir>` | Re-run saved corpus entries (JSON in DIR), one child process per entry |
| Flag | Default | Effect |
|---|---|---|
| `-a` | empty | architecture for raw input without a `_arch.s` name |
The `--fuzz` mode runs each function in a subprocess; a partial function
(e.g. a decoder that faults on malformed input) is reported as
`CRASH` without killing the parent. Use `--call` with `--buf` to invoke
partial functions with valid data instead.
With a `.s` file the file is assembled first and the listing follows the real
layout: one block per `TEXT` function, local labels printed at their offsets.
With any other file, or `-` for standard input, the bytes are disassembled
linearly and `-a` selects the architecture (amd64, arm64, riscv64 or loong64).
The `--call` mode parses the `// func` signature, allocates the requested
buffers (`zero`, `ones`, `seq`, or a hex blob), builds the ABI0 argument
block with buffer pointers/lengths/capacities at the matching parameter
offsets, and prints the arg block before and after the call, showing
return values and any output written to the buffers. Scalar parameters
are supplied with `--args` (decimal, or `0x` hex) at their ABI0 offsets.
```sh
gasm dis hello_amd64.s
```
The `--save-corpus` mode records the logical arguments (buffer contents and
scalars, not raw pointers) of every failing fuzz input as JSON. `--replay`
rebuilds a live argument block from each entry and calls it in its own child
process, reporting `OK`, `CRASH (reproduced)` or `FAIL` per entry and
exiting non-zero when any entry fails.
```text
add: 16 bytes
0000: 48 8b 44 24 08 mov rax, qword ptr [rsp+0x8]
0005: 48 03 44 24 10 add rax, qword ptr [rsp+0x10]
000a: 48 89 44 24 18 mov qword ptr [rsp+0x18], rax
000f: c3 ret
```
## `gasm debug [--func <name>] [--buf spec] [--script file] <file.s>`
## verify
Interactive debugger for JIT-assembled functions (amd64, arm64, riscv64,
loong64). Requires a compiled binary on `$PATH` (not `go run`).
```text
Usage: gasm verify [-smoke] [-abi] [-fuzz] [-ground-truth] [-profile] [-call] <file.s>
```
| Flag | Description |
|------|-------------|
| `--func` | Function to debug (required) |
| `--buf` | Buffer spec: `name:size:pattern[,name:size:pattern]` |
| `--args <file>` | File containing the ABI0 argument block |
| `--script <file>` | Run REPL commands from a file (one per line) and exit; `-` reads stdin |
| `--timeout <dur>` | Kill the debuggee after this duration (e.g. `30s`); for headless `--script` runs |
| `--cover` | Run to completion with a breakpoint on every instruction; report which executed, how often, and which labels were reached |
| Flag | Default | Effect |
|---|---|---|
| `--ground-truth` | off | compare the machine code byte-for-byte against `go tool asm` |
| `--fuzz` | off | differential fuzz against the `go tool asm` build |
| `-n` | 1000 | fuzz iterations per function |
| `--abi` | off | ABI-checking calls: sentinel registers and a red-zone canary |
| `--abi-n` | 100 | ABI check iterations with varied inputs |
| `--profile` | off | list the basic-block structure per function |
| `--smoke` | off | call each NOSPLIT function with zeroed arguments |
| `--call` | empty | invoke a single function with `--buf` instead of the sweeps |
| `--buf` | empty | buffer spec for `--call`: `name:size:pattern[,name:size:pattern]` |
| `--args` | empty | scalar args for `--call`: `name=value[,name=value]` (decimal or `0x` hex) |
| `--repeat` | 1 | number of times to repeat a `--call` invocation |
| `--save-corpus` | empty | with `--fuzz`: write each failing input to this directory as replayable JSON |
| `--replay` | empty | re-run saved corpus entries, one child process per entry |
The JIT checks run when the host matches the file's architecture; the
toolchain comparison works everywhere. `--fuzz`, `--smoke` and `--abi` run each
function in its own child process, so a partial function that faults on random
input is reported as `CRASH` instead of ending the sweep; `--call` with `--buf`
invokes such a function with valid data. loong64 stays on the ground-truth path
until hardware validation.
```sh
gasm verify --ground-truth hello_amd64.s
```
```text
hello_amd64.s: 1 functions JIT-loaded
add: MATCH (16 bytes)
ground truth: 1/1 functions byte-identical
add: 16 bytes, args=24, frame=0 NOSPLIT
```
```sh
gasm verify --call add --args a=2,b=3 hello_amd64.s
```
```text
add: 16 bytes, args=24
signature: func add(a int, b int) int
scalars:
a = 2
b = 3
args before: 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (24 bytes)
args after: 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 (24 bytes)
call 1: OK
```
## debug
```text
Usage: gasm debug <file.s> --func <name>
```
| Flag | Default | Effect |
|---|---|---|
| `-func` | empty | the function to debug, required |
| `-buf` | empty | buffer spec: `name:size:pattern[,name:size:pattern]` (zero, ones, seq or hex) |
| `-args` | empty | file containing the ABI0 argument block |
| `-script` | empty | run REPL commands from a file, one per line, and exit; `-` reads stdin |
| `-timeout` | 0 | kill the debuggee after this duration, for headless `-script` runs |
| `-cover` | off | run to completion with a breakpoint on every instruction and report which executed |
The debugger spawns the debuggee from the `gasm` binary on `$PATH`, so install
it first with `just install`; `go run` does not work for the traced child.
Requires Linux (ptrace) and all four architectures are supported.
REPL commands:
| Command | Description |
|---------|-------------|
| `break <label\|addr> [if <reg> <op> <val>]` | Set a breakpoint, optionally conditional |
| `delete <label\|addr>` | Remove a breakpoint |
| `info break` | List all breakpoints |
| `step [n]`, `s` | Single-step n instructions |
| `next`, `n` | Step over CALL |
| `finish`, `fin` | Run until the function returns |
| `continue`, `c` | Run until breakpoint, watchpoint or exit |
| `disas [n]`, `u` | Disassemble n instructions at PC |
| `regs` | Print general-purpose + vector/FP registers |
| `where` | Show source line and nearest label at PC |
| `stack` | Show stack near RSP (return address + ABI0 args) |
| `bt`, `backtrace` | Backtrace (current frame + return address) |
| `x [addr] [len]` | Hex-dump memory |
| `w <addr> <val...>` | Write bytes to memory |
| `set <reg> <value>` | Set a register |
| `watch <addr> [r\|w] [size]` | Set a hardware watchpoint (write by default) |
| `unwatch [<slot>]` | Clear one or all watchpoints |
| `labels`, `l` | List function labels and offsets |
| `help`, `h`, `?` | Show command help |
| `quit`, `q` | Kill the debuggee and exit |
| Command | Effect |
|---|---|
| `break <label\|addr> [if <reg> <op> <val>]` | set a breakpoint, optionally conditional |
| `delete <label\|addr>` | remove a breakpoint |
| `info break` | list the breakpoints |
| `step [n]`, `s` | single-step n instructions |
| `next`, `n` | step over a CALL |
| `finish`, `fin` | run until the function returns |
| `continue`, `c` | run until a breakpoint, watchpoint or exit |
| `disas [n]`, `u` | disassemble n instructions at the PC |
| `regs` | print the general-purpose and vector/FP registers |
| `where` | show the source line and the nearest label at the PC |
| `stack` | show the stack near RSP, the return address and the ABI0 args |
| `bt`, `backtrace` | backtrace: the current frame and the return address |
| `x [addr] [len]` | hex-dump memory |
| `w <addr> <val...>` | write bytes to memory |
| `set <reg> <value>` | set a register |
| `watch <addr> [r\|w] [size]` | set a hardware watchpoint, write by default |
| `unwatch [<slot>]` | clear one watchpoint or all of them |
| `labels`, `l` | list the function's labels and offsets |
| `help`, `h`, `?` | show the command help |
| `quit`, `q` | kill the debuggee and exit |
## `gasm diff [--map old=new,...] <file1.s> <file2.s>`
```sh
gasm debug --func add --cover hello_amd64.s
```
Compare the machine code produced by assembling two files. Shows which
functions differ and the first few differing bytes. Useful for verifying
that two implementations produce identical code, or for tracking encoding
changes between Go assembler versions.
## diff
| Flag | Description |
|------|-------------|
| `--map` | Comma-separated `old=new` pairs to match functions with different names |
```text
Usage: gasm diff <file1.s> <file2.s>
```
Without `--map`, functions are paired by exact name. With `--map`, a
function named `old` in the first file is compared against the function
named `new` in the second file (e.g. `--map wideCopyAVX2=wideCopyAVX512`
pairs AVX2 and AVX-512 variants regardless of suffix).
| Flag | Default | Effect |
|---|---|---|
| `-map` | empty | comma-separated `old=new` pairs to match functions with different names |
## `gasm profile <file.s>`
Functions are paired by exact name unless `--map` says otherwise, so
`--map wideCopyAVX2=wideCopyAVX512` pairs two variants regardless of suffix.
The exit status is non-zero when anything differs.
Show the basic-block structure of functions in an assembly file. Lists
each function's labels, their offsets, and the block boundaries. This is
the static structure; for runtime execution counts, use `gasm verify
--fuzz` which exercises the code paths.
```sh
gasm diff hello_amd64.s hello_amd64.s
```
## `gasm audit-instructions [amd64|arm64|riscv64|loong64]`
```text
add: identical (16 bytes)
all functions identical
```
Compare the gasm encoder for the given architecture (default amd64)
against the installed `go tool asm` and print the diff: superset
encodings (gasm-only spellings, shippable via `gasm asm --format goobj`),
known-but-unencodable names (the encoder backlog) and go-only names
(feature gaps). The Go side is probed black-box with a battery of operand
shapes per mnemonic, so the audit tracks whatever toolchain
`go env GOROOT` provides. On non-amd64 architectures the backlog is an
over-approximation: a name counts as encodable only when a probe shape
assembles cleanly, so a name whose real forms the battery misses lands
in the backlog.
## profile
## `gasm scaffold differential <file.s>`
```text
Usage: gasm profile <file.s>
```
Print a differential test skeleton for every `// func` signature in
FILE. The generated test seeds random states, drives the kernel and a
portable reference (`<name>Portable`), and compares outputs
byte-for-byte. Write the reference bodies, place the file in the
kernel's package, and run it in CI.
Show the basic-block structure of each function: its labels, their offsets and
the block boundaries. This is the static structure; for runtime execution
counts use `gasm debug --cover`, and for input coverage `gasm verify --fuzz`.
## `gasm lsp`
```sh
gasm profile hello_amd64.s
```
Run the language server over standard input/output (JSON-RPC 2.0 with
Content-Length framing). Point an LSP-capable editor at the binary and
associate it with `.s` files. The target architecture is inferred from
the file-name suffix (`_amd64.s`, `_arm64.s`, `_riscv64.s`,
`_loong64.s`).
```text
add: 16 bytes, args=24, frame=0 NOSPLIT
basic blocks: 1
```
Provides: completion, hover, document symbols, push and pull
diagnostics, semantic tokens, go-to-definition, find references, rename,
document formatting, inlay hints, code actions, signature help, document
highlights, workspace symbol search, #include document links, and
folding ranges for function bodies.
## audit-instructions
```text
Usage: gasm audit-instructions [amd64|arm64|riscv64|loong64]
```
Compare the gasm encoder for the given architecture (default amd64) against the
installed `go tool asm` and print the diff: superset encodings (gasm-only
spellings, shippable via `gasm asm --format goobj`), known-but-unencodable
names (the encoder backlog) and go-only names (feature gaps). The Go side is
probed black-box with a battery of operand shapes per mnemonic, so the audit
tracks whatever toolchain `go env GOROOT` provides. On non-amd64
architectures the backlog is an over-approximation: a name counts as encodable
only when a probe shape assembles cleanly, so a name whose real forms the
battery misses lands in the backlog.
```sh
gasm audit-instructions amd64
```
```text
gasm table (amd64, families excluded): 1542 mnemonics
gasm encodable: 580 go tool asm recognized: 1542
shared: 580
```
## scaffold
```text
Usage: gasm scaffold differential <file.s>
```
Print a differential test skeleton for every `// func` signature in FILE. The
generated test seeds random states, drives the kernel and a portable reference
(`<name>Portable`), and compares the outputs byte-for-byte. Write the reference
bodies, place the file in the kernel's package, and run it in CI.
```sh
gasm scaffold differential kernel_amd64.s > kernel_differential_test.go
```
## lsp
```text
Usage: gasm lsp
```
Run the language server over standard input/output, JSON-RPC 2.0 with
`Content-Length` framing. Point an LSP-capable editor at the binary and
associate it with `.s` files; the target architecture is inferred from the
file-name suffix (`_amd64.s`, `_arm64.s`, `_riscv64.s`, `_loong64.s`).
Provides: completion, hover, document symbols, push and pull diagnostics,
semantic tokens, go-to-definition, find references, rename, document
formatting, inlay hints, code actions, signature help, document highlights,
workspace symbol search, #include document links, and folding ranges for
function bodies. Definition, references and rename work across every open
document.
## version
```text
Usage: gasm version
```
Print the version the toolchain recorded for the build, the same string as
`gasm --version`: the tag on a tagged checkout, a pseudo-version naming the
commit below one, with `+dirty` appended on a dirty tree and `(devel)` outside
version control.
## Global flags
| Flag | Default | Effect |
|---|---|---|
| `-h`, `--help` | off | print the usage |
| `-V`, `--version` | off | print the version |
## Exit codes
| Code | Meaning |
|---|---|
| `0` | success |
| `1` | a failure the program detected: a parse or assembly error, an error-severity lint diagnostic, a mismatch in `verify`, a file that cannot be read |
| `2` | the arguments were wrong: a missing or extra argument, an unknown command or format, an invalid `--map` pair |
## Examples
Assemble a kernel, check it, and run it:
```sh
gasm lint kernel_amd64.s
gasm fmt -l kernel_amd64.s
gasm asm -o kernel.bin kernel_amd64.s
gasm verify --ground-truth kernel_amd64.s
```
Link the kernel into a Go program through the toolchain's own object format:
```sh
gasm asm --format goobj -p example.com/kernel -o kernel.o kernel_amd64.s
```
Find which labels a failing kernel reaches, headlessly:
```sh
gasm debug --func decodeBlockAVX2 --cover --script cmds.txt --timeout 30s kernel_amd64.s
```
docs/DECISIONS.md
-111
View File
@@ -1,111 +0,0 @@
# Deferred decisions
Design decisions deliberately postponed, with enough context to pick them up
again without re-deriving the analysis. Each entry records what is deferred,
why, the options on the table, and the trigger that should reopen it.
---
## GOOBJ external (cross-package) symbol references
**Status:** resolved (v0.29.0+, 2026-08-07).
**Approach taken.** Instead of parsing the compiler's iexport data (which
would have required either `golang.org/x/tools` or an in-house parser), the
resolver reads the **GOOBJ data directly** from the target package's `.a`
archive. The `.a` file contains a `_go_.o` member whose GOOBJ s is the
same one gasm writes; the parser reuses the same layout (`blkSymdef`,
`blkNonpkgdef`, the string table), so no new dependency was needed.
**How it works.**
1. `go list -json -export <pkg>` finds the target package's `.a` file.
2. `extractGOOBJ` reads the ar archive, finds the `_go_.o` member, skips
the `"go object …\n!\n"` preamble and parses the GOOBJ header.
3. `goobjFile.symbols()` walks `blkSymdef` and `blkNonpkgdef` in definition
order (the same order the linker uses) to build the symbol-to-index
mapping.
4. `resolveExternalSymbols` wires the resolved `{PkgIdx, SymIdx}` into the
GOOBJ emission.
The resolver is invoked automatically when `img.Externals` is non-empty; it
runs `go list` as a subprocess (consistent with `toolchainObjectPreamble`
which already calls `go tool asm`). All symbol data is cached per package
for the lifetime of the GOOBJ emission.
## 2026-08-30 non-amd64 JIT execution trampolines
**Status:** resolved for riscv64 (validated end to end under qemu-user)
and arm64 (fix in place, consistent with the observed frame convention);
open for loong64 until hardware validation.
**Root cause (found 2026-08-31).** The trampolines advanced SP past the
leave-address slot after loading it, while the assembled kernels read
their first argument at SP+8 per the frame convention (the amd64 path
already kept SP on that slot). Removing the advance fixed riscv64
immediately (plain and checked ABI tests pass under qemu-user); the
arm64 kernel's pre-fix trace showed exactly the same SP+8 reading. The
apparent arm64/loong64 "crashes in the JIT" turned out to be dominated
by an unrelated instability: the Go 1.26 and 1.27 runtimes crash under
qemu-user arm64 emulation (GC worker start, identical signature with the
JIT tests skipped, both qemu 7.2 and 10.2), and the Go loong64 runtime
does not start at all. `gasm verify` therefore keeps loong64 kernels on
the ground-truth path until hardware validation; the GOARCH-guarded
tests (`verify/jit_arch_test.go`, `verify/abi_arch_test.go`) are the
hardware validation entry point.
**State.** The per-architecture trampolines compile for all targets, the
kernels they execute are byte-for-byte correct against `go tool asm`, and
under `qemu-aarch64` the arm64 kernel demonstrably executes and stores its
result correctly. The failure is on the return path into Go code: arm64
and loong64 take a SIGSEGV after the kernel's RET (the Go-side unwind
through `leaveJIT` and its interposed ABIInternal wrapper is the suspect),
and riscv64 returns cleanly but with an untouched result area. amd64 is
unaffected (the checked trampoline saves and restores BP/R14 and the flow
is validated end to end).
**Evidence harness.** `verify/jit_arch_test.go` (plain call) and
`verify/abi_arch_test.go` (checked call) are GOARCH-guarded tests; build
the test binary per target (`GOARCH=arm64 go test -c -o v.test ./verify/`)
and run it under `qemu-aarch64-static` from the `verify/` directory. A
minimal reproducer pattern lives in the qemu exploration notes: verify
loads, the kernel executes, the fault follows the return.
**Fix direction.** Compare the amd64 checked trampoline (GLOBL/DATA raw
address, explicit SP/BP/R14 save-restore) against the arm64/riscv64/
loong64 `leaveJIT` unwind, in particular the interaction with the
ABIInternal wrapper that `reflect.ValueOf(leaveJIT).Pointer()` returns.
The plain-call path (no sentinels) fails the same way, so the checked
path is not the variable.
---
## 2026-08-29 tooling round
- `lint abi0-register-args`: flags kernels whose `// func` parameters are
never read from the FP frame. Motivated by a real latent bug: kernels
reading arguments from registers pass every test while the autogenerated
`F.abi0` wrapper happens to leave the caller's register values intact, and
break on a toolchain upgrade.
- `lint nonportable-register-name`: the RAX/EAX register spellings are a gasm
extension; go tool asm rejects them, so files using them only link through
the gasm goobj path.
- `lint unencodable-instruction`: a mnemonic in the architecture table that
`asm.Encodable` rejects is flagged at edit time instead of failing at
assembly time.
- `audit-instructions`: black-box diff of the encoder against go tool asm.
As of this round the tables fully overlap on names; the audit exists to
catch drift in both directions (future supersets and future gaps).
- `scaffold differential`: generates the direct-call differential skeleton
(two independent seed sets, output and in-place buffer comparison) that a
pipeline-level fuzz can never replace.
- `verify --args`: scalar arguments for `-call`, closing the repro gap where
only buffers could be supplied.
- `debug --script/--timeout/--cover`: headless debugging with a watchdog
armed before the ptrace attach (untracing sandboxes hang the attach), and
label-level block coverage for the "did my test ever enter that branch"
question.
- Superset policy remains: gasm may accept spellings and encodings go tool
asm lacks, but such kernels ship only via `gasm asm --format goobj`; the
audit reports the superset surface. The register-alias superset is warned
about by lint because the default `go build` path cannot consume it.
docs/DEVELOPMENT.md
+86 -103
View File
@@ -4,11 +4,13 @@ Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrb
## Prerequisites
- **Go** 1.27+ with `toolchain go1.27.0`
- **Go** 1.27.1, the exact version the `go` directive in `go.mod` declares
- **just**, the command runner; every task below is a just recipe
- A Linux host on amd64, arm64, riscv64 or loong64: `gasm debug` needs ptrace
and the JIT checks of `gasm verify` need executable memory
- No external dependencies beyond the Go toolchain
## Quick Start
## Setup
```sh
git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git
@@ -17,105 +19,65 @@ just build # compile bin/gasm, zero errors and zero warnings
just gates # build, fmt-check, vet, test, race: the definition of done
```
## Just Recipes
## Recipes
### `just build`
Every recipe in the `justfile`, and what it does.
Compiles `bin/gasm` with `CGO_ENABLED=0` and stripped symbols. Zero
errors, zero warnings. This is the minimum bar before any commit.
### `just gates`
The definition of done, in one command: `build`, `fmt-check`, `vet`,
`test` and `race`, in that order. Run it once per task, never per edit;
`just unit` is the one that runs after every edit.
| Recipe | What it does |
|---|---|
| `just build` | compiles `bin/gasm` with `CGO_ENABLED=0` and stripped symbols; zero errors and zero warnings |
| `just test` | the test gate: the suite with `-count=1`, the coverage profile and the 80 % floor |
| `just race` | the same suite under the race detector; the expensive one, so it runs once, inside `gates` |
| `just unit [packages] [run]` | fast, cached, scoped run for iterating: no race and no coverage, so an unchanged package reports instantly |
| `just fuzz <target> <pkg> [fuzztime]` | time-boxed fuzz of one target; the package is required, because `go test -fuzz` refuses more than one |
| `just bench [packages]` | benchmarks (`-benchmem -count=5`); on an idle machine only |
| `just fmt` | formats the tree in place with `gofmt` |
| `just fmt-check` | zero diff; prints nothing when everything is formatted, which is the shape the CI step wants |
| `just vet` | both static gates: `go vet` and `go fix -diff` |
| `just gates` | `build`, `fmt-check`, `vet`, `test` and `race`, in that order: the definition of done |
| `just clean` | removes the build artefacts, `bin/` and `coverage.out` |
| `just install` | builds, then copies the binary into `bindir` (`~/.local/bin`); `gasm debug` needs an installed binary, because it spawns the debuggee from `$PATH` |
| `just uninstall` | removes the installed binary from `bindir` |
| `just run` | runs the CLI with `go run -buildvcs=true`; the recipe takes no arguments, so flags go through the package instead |
| `just dev` | the same as `run`; the project has no watcher to add |
| `just gen` | regenerates the `arch` instruction tables from the Go toolchain source; not a gate |
### `just test`
```sh
go test -count=1 -timeout 30m -coverprofile=coverage.out \
-coverpkg=./arch/...,./asm/...,./ast/...,./disasm/...,./format/...,./lexer/...,./lint/...,./lsp/...,./parser/...,./token/...,./verify/... \
./...
go test -count=1 -timeout 10m -coverprofile=coverage.out \
./arch/... ./asm/... ./ast/... ./disasm/... ./format/... ./lexer/... \
./lint/... ./lsp/... ./parser/... ./token/... ./verify/...
```
The whole suite runs (`-count=1`, so no cached pass counts), which keeps
the packages that need hardware and the CLI glue under the gate. The
coverage floor is computed over the product packages only (arch, asm,
ast, disasm, format, lexer, lint, lsp, parser, token, verify; `debug`
traces a live process and `cmd/gasm` is CLI glue) and fails if the total
is below 80 %.
The suite runs over the logic packages (`-count=1`, so no cached pass
counts): arch, asm, ast, disasm, format, lexer, lint, lsp, parser,
token, verify. `debug` traces a live process and `cmd/gasm` is thin CLI
glue, so both sit outside the sweep, and a thin `cmd/` in it would drag
the coverage total under the floor. The floor fails if the total is
below 80 %. CI runs the same command with the same ten-minute bound, so
the number is the same everywhere.
### `just race`
The same suite under the race detector. The expensive one, so it runs
once, inside `gates`.
### `just unit`
### `just run`
```sh
just unit ./asm/ TestVexGroundTruth
just run
go run -buildvcs=true ./cmd/gasm lint kernel_amd64.s
go run -buildvcs=true ./cmd/gasm verify --ground-truth kernel_amd64.s
```
Fast, cached, scoped run for iterating: no race, no coverage, so an
unchanged package reports instantly.
### `just fuzz <target> <pkg>`
Time-boxed fuzz of one target in one package; the package is required,
because `go test -fuzz` refuses more than one. Seed corpora run as plain
tests in `unit` and `test`. Never a gate.
### `just bench`
Benchmarks (`-benchmem -count=5`). On an idle machine only.
### `just fmt`, `just fmt-check`, `just vet`
`fmt` formats in place. `fmt-check` prints nothing when everything is
formatted, which is the shape the CI step wants. `vet` runs both static
gates: `go vet` and `go fix -diff`.
### `just run -- <args>`
Runs the CLI via `go run -buildvcs=true`:
```sh
just run -- lint kernel_amd64.s
just run -- fmt -w kernel_amd64.s
just run -- verify --ground-truth kernel_amd64.s
```
### `just install`
Builds and copies the `gasm` binary into `~/.local/bin` (`BINDIR`
overrides the destination).
### `just uninstall`, `just clean`
`uninstall` removes the installed binary from `bindir`. `clean` removes
the build artefacts: `bin/` and `coverage.out`.
## Version reporting
The version is never injected. `gasm --version` prints what the
toolchain recorded in the build information: the tag on a tagged
checkout, a pseudo-version naming the commit below one, `+dirty` on a
dirty tree, and `(devel)` outside version control. There is no
`-ldflags "-X"` anywhere and no version constant in the source.
The flag on `go run` is there because it does not stamp the build otherwise,
which `--version` would then report as `(devel)`.
### `just gen`
Regenerates the architecture instruction tables in `arch/` by parsing
the Go toolchain's own assembler source
Regenerates the architecture instruction tables in `arch/` by parsing the Go
toolchain's own assembler source
(`$GOROOT/src/cmd/internal/obj/<arch>/anames.go`). Requires a Go
installation. Output is committed, with no runtime dependency on the
toolchain.
### `just uninstall`
Removes `coverage.out`, the `gasm` binary, and `*.test` artefacts.
## Running Individual Tests
## Running a single test
```sh
go test -run TestVexGroundTruth ./asm/
@@ -124,32 +86,53 @@ go test -run TestGOObjectLinkAndRun ./asm/
go test -run TestFuzzWideCopy ./verify/
```
## Debugger Note
Add `-v` for the sub-test names, and `-race` when the change touches
concurrency. `-count=1` defeats the test cache when a result looks stale.
`gasm debug` spawns a child process from the binary on `$PATH`. It does
not work with `go run`; install first:
## Coverage
```sh
just install
gasm debug --func decodeBlockAVX2 path/to/kernel_amd64.s
just test
go tool cover -func=coverage.out
```
## Project Layout
The `total:` line is the number that matters, and it stays at 80 percent or
more.
## Debugging the build
```sh
go build -gcflags='-m' ./... # inlining decisions
go build -gcflags='-S' ./... # what the compiler generated
go tool asm -S kernel_amd64.s # how the toolchain's assembler encodes a kernel
gasm dis kernel_amd64.s # what gasm makes of the same kernel
gasm tokens kernel_amd64.s # the token stream
gasm profile kernel_amd64.s # the basic blocks of each function
```
cmd/gasm/ CLI entry point (subcommands)
token/ Lexical token kinds and positions
lexer/ Hand-written scanner
ast/ Abstract syntax tree
parser/ Line-oriented parser
arch/ Register and instruction tables (generated)
lint/ Static analysis rules
format/ Canonical formatter
lsp/ Language Server Protocol server
asm/ Standalone assembler, encoder, object emitters
verify/ JIT execution, differential testing, ABI checks
debug/ Interactive ptrace debugger (all four architectures)
_gen/ Instruction table generator
testdata/ Test fixtures
docs/ Architecture, development, CLI reference
```
`gasm verify --ground-truth` is the differential check that ties the two
together: it compares gasm's bytes with `go tool asm`'s, with the relocation
sites masked, so an encoding drift shows up as a byte difference rather than a
crash later.
## Continuous integration
Workflows live in `.gitea/workflows/` and run on the project's own runners:
Test on a push or pull request to `development`, race dispatched by hand, and
the release on a `v*` tag. They are written by hand rather than through
`just`, but they enforce the same set of gates minus the race detector, which
the shared runner cannot afford on a push; a green `just gates` locally is
therefore the fastest way to a green pipeline.
## Releases
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`,
which triggers the release workflow: it builds the portable Linux targets,
takes the notes from the matching `CHANGELOG.md` section and uploads the
assets.
The version is never injected. `gasm --version` prints what the
toolchain recorded in the build information: the tag on a tagged
checkout, a pseudo-version naming the commit below one, `+dirty` on a
dirty tree, and `(devel)` outside version control. There is no
`-ldflags "-X"` anywhere and no version constant in the source.