diff --git a/CHANGELOG.md b/CHANGELOG.md index b247f9c..f5d31e7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,8 +16,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 JALR; loong64 accepts the raw `JIRL rd, rj, off` spelling the Go assembler cannot express. A frameless amd64 function containing a CALL now receives the toolchain's forced base-pointer frame. The - verify trampolines join the ground-truth lists, and a lint check for - control flow through registers and memory extends to the new forms. + riscv64 and loong64 verify trampolines join their ground-truth lists, + and a lint check for control flow through registers and memory extends + to the new forms. - **`gasm asm -GOARCH` and `gasm diff -GOARCH`.** The target architecture can be named explicitly instead of inferred from the file-name suffix, which is how the suffix-less majority of GOROOT's @@ -26,37 +27,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 file under a directory (default GOROOT/src) with the gasm encoder only: suffixed files for their architecture, suffix-less files for all four, as a GOARCH build would. Reports the headline number (127 - of 627 GOROOT files, 20.3 %, assemble for every target architecture, - against 23 in the previous release), the per-architecture pass rates - and the most common failure reasons with a representative file each, + of 627 GOROOT files, 20.3 %, assemble for every target architecture), + the per-architecture pass rates and the most common failure reasons + with a representative file each, which drive the encodability backlog by frequency. -- **Fuzz targets for the parser and the formatter.** FuzzParse (no - panic, always a usable file) and FuzzFormatIdempotency (formatting - twice equals formatting once; clean input stays clean) seed - themselves from the repository's kernels, so the plain test suite - replays every seed in CI and `just fuzz` runs the mutation engine on - demand. -- **Oracle parity as its own CI step.** The push pipeline already ran - the live go-tool-asm comparison inside the suite; a dedicated step - now names that gate when it fails. -- **Man pages.** docs/man carries gasm(1) and one page per command, - written in roff: synopsis, description, every flag with its default, - exit status, worked examples and cross-references. +- **The parser and the formatter are fuzzed.** Two targets carry the + guarantee: no input makes the parser panic, and every input yields a + file the rest of the toolkit can work on; formatting twice equals + formatting once, and clean input stays clean. They seed from the + repository's own kernels, and `just fuzz` drives the mutation engine + on demand. +- **Man pages.** docs/man carries gasm(1) and a page for every command + except `version`, which gasm(1) documents itself, written in roff: + synopsis, description, every flag with its default, exit status, + worked examples and cross-references. `just install-man` compresses them into ~/.local/share/man (MANDIR overrides) and `just uninstall-man` removes them. A test builds the - binary and compares every command's `-h` output with its page, so the - pages cannot drift from the CLI. + binary and compares each page's flags and synopsis with its own `-h` + output, so the pages cannot drift from the CLI. ### Changed +- **Go 1.27.1 required.** The module declares `go 1.27.1`, so building + from source needs that patch release or newer. - **Canonical just recipes.** `just gates` is the definition of done (build, fmt-check, vet, test, race). `install` now builds and copies the binary into `~/.local/bin` (`BINDIR` overrides) instead of downloading module dependencies, and `install-bin` is gone. The test - gate sweeps the logic packages (arch through verify; the hardware-bound - `debug` and the thin `cmd/gasm` sit outside it), so the coverage floor - is computed over the product code and the number is identical locally - and in CI. `fuzz` requires its target package. + gate sweeps the logic packages (arch through verify; the ptrace-bound + `debug` and the thin `cmd/gasm` sit outside the coverage profile), so + the coverage floor is computed over the product code and the number is + identical locally and in CI; the two excluded packages' own tests run + in the gate and in the pipelines, outside the floor. `fuzz` requires + its target package. - **The reported version comes from the build.** `gasm --version` prints the version the toolchain recorded: the tag on a tagged checkout, a pseudo-version naming the commit below one, `+dirty` on a @@ -78,8 +81,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 synopsis, the commands, every flag with its default, the exit codes and worked examples; `CONTRIBUTING.md` carries the Contributor terms and states the commit trailer form, the one-logical-change rule and the - licence header rule. The repository's own assembly (the `verify` - trampolines and the test kernels) is in `gasm fmt` canonical form. + licence header rule; `SECURITY.md` states how a vulnerability is + reported and what to expect. The repository's own assembly (the + `verify` trampolines and the test kernels) is in `gasm fmt` canonical + form. - **The README states the project's purpose and status.** It opens with a warning that the tool is an experiment under active development, version 0.x.x, free to change without warning, with 1.0.0 far off, @@ -88,7 +93,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 Go toolchain), argues the case for the syntax in a new Why Plan 9 assembly section, and carries a Direction section: extended instruction support, full GOOBJ and ELF compilation, Linux and - FreeBSD, and the four architectures. + FreeBSD, and the four architectures. A Validation status section + states what has been executed where: amd64 on real hardware, the other + three architectures under qemu-user emulation, the encoding parity on + the host for all four, and the debugger's ptrace path on amd64 only. ### Fixed @@ -125,13 +133,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 (everything after it was dropped); it is an illegal token now, invalid UTF-8 no longer inflates byte offsets, and CRLF files format to uniform LF. -- **A frameless amd64 function containing a CALL read its arguments from - the wrong stack slot.** The forced base-pointer frame shifted - FP references by eight bytes (`x+0(FP)` resolved to SP+0x18 where the - toolchain emits SP+0x10), so such functions loaded garbage. The - class-2 stack guard had the sibling defect: whenever the underflow - branch relaxed to its 32-bit form, its displacement ran four bytes - past the target and into the morestack CALL. +- **The class-2 stack guard branched four bytes past its target.** When + the underflow branch relaxed to its 32-bit form, its displacement was + still computed as if the branch were two bytes long, so it landed + inside the morestack CALL instead of the compare that decides it. + The long form is reachable once a large frame carries a body of roughly + a hundred bytes. - **Immediate operands wrapped silently on amd64.** Shift counts, immediates beyond the operand's width and displacements beyond int32 truncated without a diagnostic (`SHLQ $300` assembled as `$44`); they @@ -290,8 +297,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 the other architectures, and the abi kernels use it; every verify kernel is now ground-truth checkable (the numeric `X27` spelling the kernels used is one `go tool asm` rejects). -- The GOROOT corpus number rose to 127 of 627 files (20.3 %) assembling - for every target architecture, from 108. +- **Two more spellings GOROOT uses now assemble.** riscv64 `FCLASSD` + (classify a float64 into an integer mask) is encodable, and a + displacement written as a product (`0*8(X5)`, the toolchain's own + spelling in several kernels) parses instead of being rejected. - **loong64 JIT execution enabled.** The loong64 trampoline is now validated end to end under qemu-user emulation (plain and ABI-checked calls, goroutine-clobber detection), so `gasm verify` runs the JIT diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6e946f4..40d30ac 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -24,7 +24,9 @@ below; submitting one means you accept them. ## Development setup Requirements: Go 1.27.1, the exact version the `go` directive in `go.mod` -declares, and [just](https://github.com/casey/just) for the recipes. +declares, [just](https://github.com/casey/just) for the recipes, and a C +compiler (gcc), because `just gates` includes `just race` and the race +detector needs cgo. ```sh git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git @@ -54,16 +56,20 @@ workflow builds the assets and publishes the release and its notes. ## Code style `gofmt` and `go vet` run through `just fmt` and `just vet`, with zero diff and zero -warnings tolerated. `just gates` is the definition of done in one command, and the recipe -file names what it contains. Errors are checked explicitly, wrapped as -`fmt.Errorf("context: %w", err)`, and nothing panics outside `main`. The recipe file holds -the commands, and the language and standard-library surface is the one the `go` directive -in `go.mod` pins. +warnings tolerated. `just vet` is two gates, `go vet ./...` and `go fix -diff ./...`, +so the modernisation rewrites are enforced too. `just gates` is the definition of done in +one command, and the recipe file names what it contains. Errors are checked explicitly, +wrapped as `fmt.Errorf("context: %w", err)`, and nothing panics outside `main`. The +recipe file holds the commands, and the language and standard-library surface is the one +the `go` directive in `go.mod` pins. - `golang.org/x/arch` is the one module dependency, and it is linked into the binary: `gasm dis` and the debugger's listings decode through it. Everything else is the standard library. -- No cgo, no C, no external toolchain at runtime. +- No cgo and no C. The standalone encoder paths (`gasm asm --format raw` and `--format + elf`) need no Go installation; `gasm verify --ground-truth`, `gasm verify --fuzz`, + `gasm audit-instructions` and `gasm asm --format goobj` resolve through the installed + Go toolchain. - The parser, lexer and formatter are hand-written; the `arch` instruction tables are generated only by `_gen/gen.go` (`just gen`) and never edited by hand. - Assembly committed to the repository goes through `gasm fmt` and `gasm lint`, so a @@ -111,8 +117,8 @@ Workflows live in `.gitea/workflows/` and run on the project's own runners: | Workflow | Trigger | What it does | |---|---|---| -| Test | push or pull request to `development` | build, format check, vet, modernisation, the test suite with the coverage floor | -| Release | a `v*` tag | the same gates as Test, then the matrix build, the proven version and the release itself; the race detector runs locally in `just gates` before the tag is cut | +| Test | push or pull request to `development` | build, format check, vet, modernisation, the test suite with the coverage floor, the CLI and debugger tests outside the profile, then the oracle-parity rerun against `go tool asm` | +| Release | a `v*` tag | the same gates as Test minus the oracle-parity step, then the matrix build, the version smoke test and the release itself; the race detector runs locally in `just gates` before the tag is cut | The local equivalent is `just gates`, which is the same set plus the race detector. The race detector also has its own workflow, dispatched by hand; it never runs on a push or a diff --git a/README.md b/README.md index ae9084c..072b354 100644 --- a/README.md +++ b/README.md @@ -5,23 +5,25 @@ > output formats and behaviour can change without warning at any time. > A 1.0.0 release is light years away. Nothing in this document is a > stability promise. For all of that, this is not a paper project: gasm -> is already in active use and is tested on real assembly work. +> is already in active use and is tested on real assembly work. Only +> amd64 is validated on real hardware; the other three architectures run +> under emulation ([Validation status](#validation-status)). **GAsm** is Go's Plan 9 assembler, and Go ships it without tooling: -there is no formatter, no linter, no static analyser, no standalone -assembler and no debugger for `.s` files. Developers write assembly -blind, validate it by benchmark, and debug it by print statement. -gasm-devkit is the missing toolkit: a single, self-contained binary, -`gasm`, that serves both purposes. +there is no formatter, no linter and no debugger for `.s` files, and no +assembler that works without a Go installation. Developers write +assembly blind, validate it by benchmark, and debug it by print +statement. gasm-devkit is the missing toolkit: a single, self-contained +binary, `gasm`, that serves both purposes. - **Help develop Plan 9 assembly.** Formatting, linting, disassembly, dynamic verification, a source-level debugger and a language server, for `.s` files in Go programs. - **Use Plan 9 assembly outside the Go toolchain.** `gasm asm` encodes - on its own, with no Go installation in the loop, and writes raw - images, linkable ELF objects with DWARF5 debug sections, or the Go + on its own and writes raw images or linkable ELF objects with DWARF5 + debug sections, with no Go installation in the loop; the Go toolchain's own GOOBJ format, which `go build` consumes in place of - the toolchain's output. + the toolchain's output, needs the installed toolchain. ## Why Plan 9 assembly @@ -69,12 +71,14 @@ to give that syntax the tooling it deserves. operating recursively on directories the way `go fmt` does. `-l` lists files whose formatting differs and `-d` prints a unified diff. - **Linter.** `gasm lint` runs 18 conservative static checks, among them - `undefined-label`, `abi-argsize` (declared frame vs the `// func` signature), - `register-clobber` (Go ABI register liveness over the control-flow graph), - `stack-imbalance`, `abi0-register-args` and `unencodable-instruction`. + `undefined-label`, `abi-argsize` (declared argument area vs the `// func` + signature), `register-clobber` (Go ABI register liveness over the + control-flow graph), `stack-imbalance`, `abi0-register-args` and + `unencodable-instruction`. - **Standalone assembler.** `gasm asm` encodes all four architectures without - the Go toolchain and writes raw images, linkable ELF objects (with DWARF5 - debug sections) or the Go toolchain's own GOOBJ format, which `go build` + the Go toolchain and writes raw images or linkable ELF objects (with DWARF5 + debug sections) with no Go installation needed, or the Go toolchain's own + GOOBJ format, which needs the installed toolchain and which `go build` consumes in place of the toolchain's output. Framed functions get the stack-split guard and the morestack block, byte-identical to the toolchain's, so split functions link too. @@ -86,7 +90,8 @@ to give that syntax the tooling it deserves. byte-for-byte ground-truth comparison of the machine code. - **Debugger.** `gasm debug` is a source-level ptrace debugger with breakpoints (optionally conditional), hardware watchpoints, register and - memory inspection, and headless script runs with label-level coverage. + memory inspection, and headless script runs that report instruction and + label coverage. - **Language server.** `gasm lsp` serves completion, hover, document symbols, push and pull diagnostics, semantic-token highlighting, go-to-definition, find references, rename, formatting, inlay hints, code actions, signature @@ -123,6 +128,34 @@ The same measurement runs over GOROOT's whole assembly corpus: assembling for every target architecture today, with the top failure reasons per architecture; the number moves with every release. +### Validation status + +**Only amd64 is validated on real hardware.** The other three +architectures are validated under qemu-user emulation, because the +project owns no arm64, riscv64 or loong64 machine, and emulation is the +only substitute available for the hardware. The distinction matters and +is stated rather than implied: everything below is a claim about what has +actually been executed. + +| Layer | amd64 | arm64, riscv64, loong64 | +|---|---|---| +| Encoding: byte-for-byte against `go tool asm` | native hardware | native hardware (the toolchain cross-assembles any GOARCH on any host) | +| Execution: JIT calls, ABI checks, differential fuzzing | native hardware | qemu-user emulation | +| Debugger: ptrace tracing, breakpoints, watchpoints, coverage | native hardware | emulation cannot run ptrace; the layer compiles and its architecture-neutral units run under `go test ./...`, nothing more | + +Consequences, stated plainly. An emulator is a model of a CPU, not the +CPU: instruction semantics are implemented in software and can differ +from silicon in ways a test suite does not reveal. A kernel that passes +under qemu-user is therefore not proven correct on real hardware, and a +discrepancy found on real hardware is a defect in gasm, reported like any +other. Encoding parity is the exception: the byte comparison against the +toolchain runs on the host for every architecture, so no emulator stands +between the claim and the evidence. The debugger is the weakest case: on +the three emulated architectures its per-architecture ptrace code has +been compiled and read, never executed. Its architecture-neutral units +run under `go test ./...`, which the race workflow and a manual run +perform; the default `just test` gate does not sweep `./debug/...`. + ## Direction The plan, in the order it is being worked: @@ -134,7 +167,8 @@ The plan, in the order it is being worked: an extended instruction set the toolchain does not know at all. The toolchain-derived tables stay generated and untouched; only the extended instructions are hand-maintained, with their own spellings - and encoders, verified by execution on real hardware because the + and encoders, verified by execution (on real hardware for amd64, under + emulation for the rest, per the validation status above) because the toolchain offers no ground truth to compare against. The gaps exist on every architecture, amd64 included. - **Full GOOBJ and ELF compilation.** The destination is a complete, @@ -205,7 +239,7 @@ gasm verify --ground-truth k.s # byte-for-byte vs go tool asm gasm verify --fuzz k.s # differential fuzz vs the go tool asm build gasm debug --func name k.s # interactive debugger gasm debug --func name --script cmds.txt --timeout 30s k.s # headless run -gasm debug --func name --cover k.s # which labels did execution reach? +gasm debug --func name --cover k.s # instruction and label coverage gasm diff a.s b.s # compare machine code byte-for-byte gasm diff --map wideCopyAVX2=wideCopyAVX512 avx2.s avx512.s gasm profile k.s # show basic-block structure @@ -243,7 +277,8 @@ recipe. - [docs/CLI.md](docs/CLI.md): full command reference - man pages: `just install-man` installs gasm(1) and one page per command - into ~/.local/share/man (MANDIR overrides); `just uninstall-man` removes + except `version`, which is documented inside gasm(1) instead, into + ~/.local/share/man (MANDIR overrides); `just uninstall-man` removes them - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): components and data flow - [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md): development setup and recipes diff --git a/SECURITY.md b/SECURITY.md index f391217..adf687b 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -29,7 +29,8 @@ Include: - You are kept informed while the fix is being made, and told when it ships. - The fix is released before the details are published, and the timing is agreed with you. -- The reporter is credited in the release notes unless they ask otherwise. +- The fix ships without naming you: the project keeps no credits list, so the release + notes, the changelog and the commits name no reporter. ## Out of scope diff --git a/cmd/gasm/audit.go b/cmd/gasm/audit.go index 958cb7a..3b8ca6b 100644 --- a/cmd/gasm/audit.go +++ b/cmd/gasm/audit.go @@ -40,10 +40,13 @@ func cmdAuditInstructions(args []string) error { fs := newCommand("audit-instructions", "gasm audit-instructions [--corpus [dir]] [amd64|arm64|riscv64|loong64]", ` Compare the gasm encoder for the given architecture (default amd64) against go tool asm and print the diff: superset encodings (gasm-only, shippable via -gasm asm --format goobj), known-but-unencodable names (the backlog) and go- -only names (feature gaps). The Go side is probed black-box with a battery -of bare mnemonics, so the audit tracks whatever toolchain `+"`go env GOROOT`"+` -provides. +gasm asm --format goobj) and known-but-unencodable names (the backlog). The +Go side is probed black-box one bare mnemonic at a time, so the audit tracks +whatever toolchain `+"`go env GOROOT`"+` provides; the gasm side answers from +the encoder table on amd64 and from trial assembly over a battery of operand +shapes elsewhere. Names go tool asm knows and gasm does not cannot be +enumerated by probing, because Go's table is visible only through names +already in the gasm table; the report closes with a note saying so. With --corpus the audit changes shape: it assembles every .s file under the given directory (default GOROOT/src) with the gasm encoder only, no @@ -109,7 +112,7 @@ the encodability backlog by frequency rather than by table order. w := os.Stdout fmt.Fprintf(w, "gasm table (%s, families excluded): %d mnemonics\n", archName, len(names)) - fmt.Fprintf(w, "gasm encodable: %d go tool asm recognized: %d\n", len(shared)+len(superset), countTrue(goKnown)) + fmt.Fprintf(w, "gasm encodable: %d go tool asm recognised: %d\n", len(shared)+len(superset), countTrue(goKnown)) fmt.Fprintf(w, "shared: %d\n", len(shared)) fmt.Fprintf(w, "\nSuperset encodings (gasm-only; ship via gasm asm --format goobj):\n") for _, n := range superset { diff --git a/cmd/gasm/debug_linux.go b/cmd/gasm/debug_linux.go index afe5a04..06ca91e 100644 --- a/cmd/gasm/debug_linux.go +++ b/cmd/gasm/debug_linux.go @@ -24,9 +24,10 @@ function in a traced subprocess (ptrace), then provides a REPL for single-stepping, breakpoints, register and memory inspection. REPL commands: - break [if ] + break [if ] set a breakpoint, optionally conditional on a - register comparison (reg-reg or reg-immediate) + comparison of one register against a constant, + another register, or the 8-byte word at *addr delete remove a breakpoint info break list all breakpoints step [n], s single-step n instructions (default 1) diff --git a/cmd/gasm/main.go b/cmd/gasm/main.go index 0e9ca3e..a5fa4c2 100644 --- a/cmd/gasm/main.go +++ b/cmd/gasm/main.go @@ -486,10 +486,13 @@ func cmdAsm(args []string) int { With -o the output is written to a file instead. The --format flag selects what is written: raw (the default) concatenates the functions and the data section into one self-consistent image; elf emits a relocatable object -(.text/.data sections, a symbol table and one PC32 relocation per -static-symbol reference) that links with the system toolchain; goobj emits -the Go toolchain's own object format, which cmd/link consumes directly (it -requires -p, the package path, and the installed Go toolchain). +(.text/.data sections, a symbol table and one relocation per static-symbol +reference, in the architecture's own form: R_X86_64_PC32 on amd64, +R_AARCH64_*, R_RISCV_* or R_LARCH_* on the others) that links with the +system toolchain; goobj emits the Go toolchain's own object format, which +cmd/link consumes directly (it requires -p, the package path, and the +installed Go toolchain: the object preamble is captured from go tool asm +and the format version from go version). `) out := fs.String("o", "", "write the output to this file") format := fs.String("format", "raw", "output format: raw (concatenated image), elf or goobj (Go object)") @@ -793,8 +796,8 @@ func cmdProfile(args []string) int { flagSet := newCommand("profile", "gasm profile ", ` Show the basic-block structure of functions in an assembly file. Lists each function's labels, their offsets, and the block boundaries. -This is the static structure; for runtime execution counts, use -gasm verify --fuzz which exercises the code paths. +This is the static structure; for runtime execution counts use +gasm debug --cover, and for input coverage gasm verify --fuzz. `) flagSet.Parse(args) if flagSet.NArg() != 1 { @@ -1033,7 +1036,8 @@ With -profile, the static basic-block structure is listed for each function. With -call, a single function is invoked with user-supplied buffers (-buf) instead of the smoke/abi/fuzz sweeps. Useful for partial functions (e.g. -decoders) that crash on random input but should succeed on valid data. +decoders) that crash on random input but should succeed on valid data. The +function named must be NOSPLIT: a function with a stack frame is refused. With -save-corpus (and -fuzz), every input that crashes or mismatches is written to the directory as replayable JSON. -replay re-runs saved diff --git a/debug/repl.go b/debug/repl.go index ad60881..cdde45d 100644 --- a/debug/repl.go +++ b/debug/repl.go @@ -33,7 +33,7 @@ func REPL(s *Session, bm *Breakpoints, codeBase uint64, funcOffset, funcSize, ar entryAddr := codeBase + uint64(funcOffset) fmt.Printf("stopped at function entry: %#x (%d bytes)\n", entryAddr, funcSize) - fmt.Println("commands: break | step [n] | continue | disas [n] | regs | where | x [len] | w | labels | quit") + fmt.Println("commands: break | step [n] | continue | disas [n] | regs | where | x [len] | w | labels | quit") scanner := bufio.NewScanner(in) @@ -233,7 +233,7 @@ func REPL(s *Session, bm *Breakpoints, codeBase uint64, funcOffset, funcSize, ar case "break", "b": if len(parts) < 2 { - fmt.Println("usage: break [if ]") + fmt.Println("usage: break [if ]") continue } var addr uint64 @@ -278,7 +278,7 @@ func REPL(s *Session, bm *Breakpoints, codeBase uint64, funcOffset, funcSize, ar } } } else if len(parts) >= 4 && parts[2] == "if" { - fmt.Println("usage: break if ") + fmt.Println("usage: break if ") continue } bp, err := bm.SetWithCond(addr, label, cond) @@ -422,7 +422,7 @@ func REPL(s *Session, bm *Breakpoints, codeBase uint64, funcOffset, funcSize, ar fmt.Println() case "help", "h", "?": - fmt.Printf(` break [if ] + fmt.Printf(` break [if ] set a breakpoint, optionally conditional on a register compared to a constant, a register, or the 8-byte word at *addr diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index dc0ecf8..1826b32 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -13,11 +13,13 @@ Three design goals shape everything below. So the centre of the toolkit is a hand-written lexer and a parser that produce a typed AST with source positions on every node. 2. **Architecture as data, not code.** Per-architecture differences (amd64, - arm64, riscv64, loong64) live in register and instruction *tables* (`arch`), - never in `if arch == …` branches scattered through the logic. The - instruction tables are generated from the Go toolchain's own assembler - source (`just gen`), so adding or refreshing an architecture is a data - operation, not a coding one. + arm64, riscv64, loong64) live in register and instruction *tables* (`arch`) + and per-architecture encoders, rather than in `if arch == …` branches + threaded through the analysis; the arch tests that remain are dispatch and + policy points, such as which encoder a file name selects and which + registers the liveness pass audits. The instruction tables are generated + from the Go toolchain's own assembler source (`just gen`), so refreshing an + architecture is a data operation, not a coding one. 3. **Open integration surface.** Everything the toolkit can do is reachable through two vendor-neutral interfaces: a CLI and an LSP server. No editor owns the toolkit; the toolkit is offered to editors on standard terms. @@ -35,10 +37,19 @@ flowchart TD ARCH["arch tables
amd64 / arm64 / riscv64 / loong64"] --> LINT ARCH --> LSP LINT --> LSP + PAR --> ASM["asm
encoders, image, object emitters"] + ASM --> VER["verify
JIT mapping, ABI checks, fuzzing"] + ASM --> DBG["debug
ptrace session"] + VER --> DBG + DIS["disasm
golang.org/x/arch"] --> DBG FMT --> CLI["gasm CLI"] LINT --> CLI PAR --> CLI LEX --> CLI + ASM --> CLI + VER --> CLI + DBG --> CLI + DIS --> CLI LSP --> EDITOR["any LSP editor"] ``` @@ -70,9 +81,11 @@ assembler provides. The boundaries matter as much as the responsibilities: `ast` records syntax only, and whether a name is a register or a label is left to `arch`, so the -parser stays architecture-agnostic. `asm` and `verify` are the only packages -that touch machine code and executable memory, and `cmd/gasm` owns no logic -beyond flags and output. +parser stays architecture-agnostic. `asm` produces the machine code, `verify` +and `debug` are the two packages that map it executable (read-execute in +`verify`, read-write-execute in the debuggee), and `cmd/gasm` is the CLI, with +the verify sweep orchestration and the audit, scaffold and unified-diff +helpers beside its flags and output. ### `token` and `lexer` @@ -112,14 +125,17 @@ Register files are generated programmatically (the regular `R8`-`R15`, `X0`-`X15`, `Y0`-`Y15`, `Z0`-`Z31`, `K0`-`K7` ranges) plus the irregularly named registers listed explicitly. Instruction names are **generated from the Go toolchain's own assembler source** (`cmd/internal/obj//anames.go`, -plus the common opcodes and the per-architecture front-end aliases such as the -arm64 `B`/`BL` branches and the `.P`/`.W` load-store addressing suffixes) by -`just gen`, so the tables always match what the real assembler accepts. Each -mnemonic maps to a summary and an optional operand-count range; counts are -recorded only where unambiguous (`-1` disables the operand-count lint for that -instruction) so the linter stays silent rather than guess. For architectures -with highly variable operand forms (arm64, riscv64, loong64) only a few -fixed-arity instructions (`RET`, `NOP`, `JMP`, `CALL`) carry counts at all. +plus the common opcodes in `cmd/internal/obj/util.go`) by `just gen`, so the +tables always match what the real assembler accepts. The spellings the +toolchain's tables do not carry are hand-maintained instead: the front-end +alias lists in `arch/arm64.go`, `arch/amd64.go` and `arch/loong64.go` (the +arm64 `B`/`BL` branches among them), and the arm64 `.P`/`.W` load-store suffix +stripping in `arch/arch.go`. Each mnemonic maps to a summary and an optional +operand-count range; counts are recorded only where unambiguous (`-1` +disables the operand-count lint for that instruction) so the linter stays +silent rather than guess. For architectures with highly variable operand +forms (arm64, riscv64, loong64) `relaxCounts` clears those counts, leaving +`RET` and `NOP` with a range (`RET` alone on riscv64). ### `lint` @@ -291,11 +307,11 @@ registers are translated onto the hardware stack pointer: `x+N(FP)` becomes pointer is set up, with the matching Go prologue/epilogue generated, so the output is byte-identical to the Go assembler for these cases. SIMD is handled by a VEX (AVX/AVX2) encoder (the two- and three-byte VEX prefixes with XMM/YMM -registers) across eight operand forms: the three-operand NDS form, the -two-operand reg/rm form, the immediate-shift form (plus the variable-count -shifts, which share the NDS shape with the count in an XMM register or -memory), the immediate shuffle form (`VPSHUFD`, `VPERMQ`), the -three-operand-plus-immediate form (`VSHUFPD`, +registers) over nine operand forms plus a dedicated move encoder: the +three-operand NDS form, the two-operand reg/rm form, the immediate-shift form +(plus the variable-count shifts, which share the NDS shape with the count in +an XMM register or memory), the immediate shuffle form (`VPSHUFD`, `VPERMQ`), +the three-operand-plus-immediate form (`VSHUFPD`, `VPERM2I128`, `VINSERTI128`), the lane-extract form (`VEXTRACTI128`, `VEXTRACTF128`, where the YMM source occupies the reg field and the XMM or memory destination r/m), the direction-sensitive moves (`VMOVDQU`, `VMOVUPD`, @@ -340,10 +356,10 @@ b bit and the L'L rounding-control field (broadcast keeps the vector length and scales disp8 by the element size), and combine with the .Z zeroing suffix. Every encoding is validated two ways: by round-trip decoding through `golang.org/x/arch`, and byte-for-byte against -the machine code the real Go assembler emits, a comparison that holds for -whole functions: all 27 functions of both kernels assemble to exactly the Go -toolchain's bytes, the lone exception being the displacements of the -static-constant loads, which the Go linker fills at link time. +the machine code the real Go assembler emits; the parity suites carry that +comparison over whole kernel files on all four architectures, with the +relocation fields masked because the Go linker fills those displacements at +link time. File-level assembly (`AssembleFile`) goes beyond single functions: it materialises the file's static symbols (`GLOBL`/`DATA`) in a data section @@ -426,10 +442,13 @@ The `gasm verify` CLI subcommand exposes this: it loads a file, reports the available functions and (with `-smoke`) calls each NOSPLIT function with zeroed arguments to confirm the trampoline round-trips. The `-smoke` and `-abi` sweeps run in parallel and each inside a child process, so a function that -faults is reported without ending the sweep. `gasm verify --fuzz` combines -ABI checks (sentinel registers, canary, stack bounds) with differential fuzz -testing, comparing the JIT-assembled kernel against the portable Go reference -bit-for-bit while verifying the ABI contract on every iteration. When a fuzz +faults is reported without ending the sweep; `-abi` is where the ABI check +lives, fuzzing each function with sentinel values in the registers the Go ABI +fixes across calls and a canary below `SP`, and reporting a violation on any +iteration. `gasm verify --fuzz` is the differential campaign instead: it +JIT-loads the kernel and the `go tool asm` build of the same kernel and +compares the output argument areas bit-for-bit, one child process per function +so a crash on a partial function is reported rather than fatal. When a fuzz iteration crashes or mismatches, `FuzzResult.CrashInput` stores the exact input for reproducibility. `gasm verify --call --buf name:size:pattern` invokes a single function with user-supplied buffers (patterns: zero, ones, @@ -449,16 +468,25 @@ masked), reporting any encoding drift. The interactive debugger (all four architectures). It launches the target function in a child process that maps the JIT code, calls `PTRACE_TRACEME`, and stops; the parent attaches via ptrace and controls -execution. Breakpoints are patched as INT3 bytes through `/proc/pid/mem` -(PTRACE_PEEKTEXT is unreliable with Go's multi-threaded runtime). +execution. Breakpoints are patched through `/proc/pid/mem`: the one-byte +`INT3` on amd64, the four-byte break instruction on the other three (arm64 +`BRK #0`, riscv64 `ebreak`, loong64 `break 0`). The child pins its goroutine to the OS thread with `runtime.LockOSThread` so the traced thread is the one executing JIT code. The REPL provides -single-step, register inspection (GPR + YMM/XMM via `PTRACE_GETFPREGS`), +single-step, register inspection (the GPRs on every architecture; on amd64 the +XMM set through `PTRACE_GETFPREGS` and the YMM set through `PTRACE_GETREGSET` +on `NT_X86_XSTATE`; on the other three the FP/SIMD regset through +`PTRACE_GETREGSET` on `NT_PRFPREG`), label resolution, named buffer allocation with pattern filling (`--buf name:size:pattern`: zero, ones, seq, or hex), and breakpoint management. Breakpoints accept conditions (`break