From c834d98210faca070015b11a73fdf074c6a33f84 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petr=20Balv=C3=ADn?= Date: Thu, 17 Sep 2026 20:33:18 +0200 Subject: [PATCH] docs: bring the document set into the standard shape Assisted-by: GLM 5.3 Flash --- CHANGELOG.md | 38 ++- CONTRIBUTING.md | 176 ++++++++------ README.md | 3 +- docs/ARCHITECTURE.md | 111 ++++++++- docs/CLI.md | 549 ++++++++++++++++++++++++++++++------------- docs/DECISIONS.md | 111 --------- docs/DEVELOPMENT.md | 189 +++++++-------- 7 files changed, 700 insertions(+), 477 deletions(-) delete mode 100644 docs/DECISIONS.md diff --git a/CHANGELOG.md b/CHANGELOG.md index bcd1c04..df4b929 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ All notable changes to gasm-devkit are documented here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), -and this project adheres to [Conventional Commits](https://www.conventionalcommits.org/). +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [development] @@ -13,20 +13,42 @@ and this project adheres to [Conventional Commits](https://www.conventionalcommi (build, fmt-check, vet, test, race). `install` now builds and copies the binary into `~/.local/bin` (`BINDIR` overrides) instead of downloading module dependencies, and `install-bin` is gone. The test - gate sweeps the whole suite and computes the coverage floor over the - product packages with `-coverpkg`, `disasm` now included, so the - number is identical locally and in CI. `fuzz` requires its target - package. + gate sweeps the logic packages (arch through verify; the hardware-bound + `debug` and the thin `cmd/gasm` sit outside it), so the coverage floor + is computed over the product code and the number is identical locally + and in CI. `fuzz` requires its target package. - **The reported version comes from the build.** `gasm --version` prints the version the toolchain recorded: the tag on a tagged checkout, a pseudo-version naming the commit below one, `+dirty` on a dirty tree and `(devel)` outside version control. Nothing is injected with `-ldflags -X` any more. - **CI realigned with the gate set.** The push pipeline runs the gates - minus race in one job, with a cached Go setup and the module as the - version source; the race detector moved to a hand-dispatched workflow - and into the release gates; the release builds without injection and + minus race in one job, in the `gates` order, with a cached Go setup and + the module as the version source; a superseded run of the same branch + is cancelled instead of queueing; every `go test` runs under a + ten-minute bound that matches its job's; the race detector moved to a + hand-dispatched workflow and runs in the local gate before a tag is + cut, never on a push or a tag; the release builds without injection and its smoke test requires the recorded tag and rejects `+dirty`. +- **The documents follow the standard set.** `docs/ARCHITECTURE.md` is + organised as Overview, Packages, Data flow, State and lifetime and + Dependencies, and carries a sequence diagram of the assembly path; + `docs/DEVELOPMENT.md` lists every recipe in one table and documents the + coverage floor, the CI and the release flow; `docs/CLI.md` gives the + synopsis, the commands, every flag with its default, the exit codes and + worked examples; `CONTRIBUTING.md` carries the Contributor terms and + states the commit trailer form, the one-logical-change rule and the + licence header rule. The repository's own assembly (the `verify` + trampolines and the test kernels) is in `gasm fmt` canonical form. + +### Fixed + +- **The dependency statement was wrong.** `golang.org/x/arch` is not + test-only: `gasm dis` and the debugger's listings decode through it, so + it is linked into the binary. `CONTRIBUTING.md` and + `docs/ARCHITECTURE.md` said otherwise. +- **The CLI reference listed 17 of the 18 lint rules.** The missing + `reserved-register-write` is documented with the rest. ## [0.33.0] - 2026-09-14 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e9024b4..d50e062 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,107 +1,127 @@ -# Contributing to gasm-devkit +# Contributing -Thanks for contributing to gasm-devkit. +Contributions to **gasm-devkit** are governed by the Contributor terms +below; submitting one means you accept them. + +## Contributor terms + +1. This project belongs to its owner alone. The owner decides what is + accepted, in what form and when; the decision is final and needs no + justification. +2. By submitting a contribution you assign to Petr Balvín + all present and future copyright and + related rights in it, worldwide, for the full term of the rights, + with the right to relicense and sublicense without restriction, + including under proprietary terms. +3. Where that assignment is not effective, it counts as a perpetual, + irrevocable, royalty-free licence with the same scope. +4. To the fullest extent permitted by law, you waive any right of + attribution and integrity in the contribution. The project names no + contributors and keeps no credits list. +5. By submitting you represent that the work is yours and that you + hold the rights to assign it as above. ## Development setup -Requirements: Go 1.27 or later, the [just](https://github.com/casey/just) -command runner, and a Linux host on amd64, arm64, riscv64 or loong64. +Requirements: Go 1.27.1, the exact version the `go` directive in `go.mod` +declares, and [just](https://github.com/casey/just) for the recipes. ```sh git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git cd gasm-devkit -just build # compile, zero errors and zero warnings -just gates # build, fmt-check, vet, test, race: the definition of done +just build +just gates ``` ## Workflow -1. Branch from `development`; never commit directly to `main` (`main` is - release-only: merge from `development`, then tag). -2. Commit with [Conventional Commits](https://www.conventionalcommits.org/): - `type(scope): description`: subject line only, imperative mood, - lowercase after the colon, no trailing dot. Allowed types: `feat`, - `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`, `ci`, - `build`, `revert`. The only line after the subject is the trailer: - `Assisted-by: `. No `Co-Authored-By`, no `Signed-off-by`, - no other trailers. -3. Record every user-visible change in `CHANGELOG.md` under - `## [development]` (categories: Added, Changed, Fixed, Removed, - Security). -4. Add or update tests; coverage must stay **at or above 80 %** (hard - gate, enforced by CI). -5. Update the documentation when behaviour, flags or the public surface - change. -6. Open a pull request against `development`. +1. Branch from `development`. Never commit directly to `main`, which is release-only. +2. Commit in [Conventional Commits](https://www.conventionalcommits.org/) form: + `type(scope): description`, subject line only, imperative mood, lowercase after the + colon, no trailing full stop. Allowed types: `feat`, `fix`, `docs`, `style`, + `refactor`, `perf`, `test`, `chore`, `ci`, `build`, `revert`. +3. One logical change per commit. A refactor, a behaviour change and a formatting pass + are three commits, never one. +4. Record every user-visible change in `CHANGELOG.md` under `## [development]`. +5. Add or update tests. Coverage stays at 80 percent or more; it is a hard gate. +6. Update the documentation when the public API, the configuration or the behaviour + changes. +7. Open a pull request against `development`. -Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`; -CI builds and publishes the binaries for all four architectures. +Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The release +workflow builds the assets and publishes the release and its notes. ## Code style -`gofmt` and `go vet` via `just fmt` / `just vet`; both must pass with -zero output; `go fix -diff ./...` must report nothing on touched packages. +`gofmt` and `go vet` run through `just fmt` and `just vet`, with zero diff and zero +warnings tolerated. `just gates` is the definition of done in one command, and the recipe +file names what it contains. Errors are checked explicitly, wrapped as +`fmt.Errorf("context: %w", err)`, and nothing panics outside `main`. The `golang` +skill holds the rules the project follows; the recipe file holds the commands. -- Standard library only in production code; `golang.org/x/arch` is used - in tests only (round-trip decoding) and is never linked into the `gasm` - binary. -- No cgo, no C, no external toolchains at runtime. -- Explicit `if err != nil`; errors wrapped with - `fmt.Errorf("context: %w", err)`; no panics outside `main`. -- The parser, lexer and formatter are hand-written; the `arch` instruction - tables are generated only via `_gen/gen.go` (`just gen`), never edited. +- `golang.org/x/arch` is the one module dependency, and it is linked into the binary: + `gasm dis` and the debugger's listings decode through it. Everything else is the + standard library. +- No cgo, no C, no external toolchain at runtime. +- The parser, lexer and formatter are hand-written; the `arch` instruction tables are + generated only by `_gen/gen.go` (`just gen`) and never edited by hand. +- Assembly committed to the repository goes through `gasm fmt` and `gasm lint`, so a + `.s` file that `gasm fmt -l .` lists is unfinished. -## Running a single test +New source files open with the project's two-line licence header, whose SPDX +identifier matches `LICENSE`. Configuration files, workflows and dotfiles do not carry +it. -```sh -go test -run TestVexGroundTruth ./asm/ -go test -run TestGroundTruthBasic ./verify/ -go test -run TestGOObjectLinkAndRun ./asm/ -go test -run TestFuzzWideCopy ./verify/ -``` +## AI contribution policy -The interactive debugger (`gasm debug`) requires a compiled binary on -`$PATH`; `go run` does not work for the traced child process. Install -first with `just install`. +AI tools are welcome as productivity aids and are a normal part of modern software +development. What matters is that the contribution stays understandable, reviewable and +genuinely useful. -## CI (Gitea Actions) +- **Disclose the assistance.** If AI helped draft any part of a commit, issue, pull + request or review, say so. +- **Commit messages carry exactly one trailer**, as a git trailer on the line after a + blank line that closes the subject: -Workflows live in `.gitea/workflows/` and run on self-hosted runners: + ``` + Assisted-by: MODEL + ``` + + Name the model that did the work, spelled the way its maker spells it, for example + `GLM 5.3`, `DeepSeek V4.1 Flash` or `Qwen 3.8 Flash`. No `Co-Authored-By`, no `Signed-off-by`, + no other trailers, and no prose: the trailer is the disclosure. +- **Issues and pull requests** attribute the assistance in a comment, for example + `_Assisted-by: GLM 5.3_`. It does not belong in the pull request description. +- **Take responsibility.** You are accountable for the accuracy, completeness and + intent of everything you submit, whether or not AI produced it. +- **Review before marking ready.** Read the diff carefully, run it locally, and add the + tests it needs. Do not mark a pull request ready until you can defend every change in + it. +- **Quality over quantity.** Contributions that look like un-reviewed output, or whose + author cannot engage substantively during review, may be closed. +- **Preferred models.** Prefer open-weight models with transparent training data and + minimal output filtering. + +AI assists. It does not replace judgement. + +## Continuous integration + +Workflows live in `.gitea/workflows/` and run on the project's own runners: | Workflow | Trigger | What it does | -|----------|---------|--------------| -| Test | push / PR to `development` | the `gates` set minus race: gofmt, `go vet`, `go fix`, build, the suite, the 80 % coverage floor | -| Race | manual dispatch, and inside the release | the suite under the race detector | -| Release | tag `v*` | the gates once, then cross-compiles binaries for linux/{amd64,arm64,riscv64,loong64} and publishes the Gitea release | +|---|---|---| +| Test | push or pull request to `development` | build, format check, vet, modernisation, the test suite with the coverage floor | +| Release | a `v*` tag | the same gates as Test, then the matrix build, the proven version and the release itself; the race detector runs locally in `just gates` before the tag is cut | -The Definition of Done (`just gates`) must still pass locally before -pushing. - -## AI Contribution Policy - -AI tools are welcome as productivity aids. What matters is that -contributions remain understandable, reviewable, and genuinely useful. - -- **Disclose AI use.** If you used AI to draft or generate any part of a - commit, issue, pull request, or code review, say so clearly. -- **Commit messages:** end every commit with exactly one trailer: - `Assisted-by: ` (e.g. `Assisted-by: GLM 5.3`). -- **Pull requests and issues:** attribute AI assistance in one trailing - line, e.g. `_Assisted-by: GLM 5.3_`. Do not paste it into the PR - description as a section. -- **Take responsibility.** You remain accountable for the accuracy, - completeness, and intent of everything you submit. -- **Review before marking ready.** Read AI-generated diffs carefully, run - them locally, and add or update tests where appropriate. -- **Preferred models.** Prefer open-weight models with transparent - training data: **GLM**, **DeepSeek**, and **MiMo**. +The local equivalent is `just gates`, which is the same set plus the race detector. The +race detector also has its own workflow, dispatched by hand; it never runs on a push or a +tag, where it would double the time and the memory a shared runner cannot spare. ## Reporting bugs -Open an issue at -[sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit/issues) -with the version (`gasm --version`), OS and architecture, the exact -command, the full output, and the expected versus actual behaviour. +Open an issue at `https://sourcedock.dev/petrbalvin/gasm-devkit/issues` with the +version, the operating system and architecture, the exact command, the full output, +and the expected against the actual behaviour. -**Security issues:** email **opensource@petrbalvin.org** instead of opening -a public issue. +**Security issues do not go in the issue tracker.** Report them as +[SECURITY.md](SECURITY.md) describes, to **opensource@petrbalvin.org**. diff --git a/README.md b/README.md index 2f0aa6e..e543930 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ has no runtime dependency on the toolchain. Prebuilt binaries for linux/amd64, linux/arm64, linux/riscv64 and linux/loong64 are on the [releases page](https://sourcedock.dev/petrbalvin/gasm-devkit/releases). -From source (Go 1.27 or later): +From source (Go 1.27.1): ```sh go install sourcedock.dev/petrbalvin/gasm-devkit/cmd/gasm@latest @@ -161,7 +161,6 @@ recipe. - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): components and data flow - [docs/CLI.md](docs/CLI.md): full command reference - [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md): development setup and recipes -- [docs/DECISIONS.md](docs/DECISIONS.md): deferred design decisions - [CHANGELOG.md](CHANGELOG.md): release history ## Licence diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5da1378..5fd90c8 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -4,7 +4,9 @@ How gasm-devkit is put together and why. Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit) -## Design goals +## Overview + +Three design goals shape everything below. 1. **A real AST, not a grammar hack.** The linter, analyser, assembler and language server all need to *reason* about assembly, not just colour it. @@ -20,10 +22,10 @@ Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrb through two vendor-neutral interfaces: a CLI and an LSP server. No editor owns the toolkit; the toolkit is offered to editors on standard terms. -## Pipeline +The components, and how data moves between them: ```mermaid -graph TD +flowchart TD SRC["source .s"] --> LEX["lexer
token stream"] LEX --> PAR["parser
AST + diagnostics"] LEX --> FMT["format
re-space tokens"] @@ -42,9 +44,34 @@ graph TD The lexer is the shared foundation: the parser builds the AST from it, the formatter re-spaces its tokens directly, and the language server uses it for -semantic highlighting. +semantic highlighting. The phases follow a dependency chain: Phase 1 (static +analysis) builds only on the AST, Phase 2 (the standalone assembler) emits +object code, and Phases 3 (dynamic analysis) and 4 (the debugger) both consume +the execution substrate that the assembler provides. -## Components +## Packages + +| Package | Responsibility | +|---|---| +| `token` | token kinds and positions | +| `lexer` | hand-written scanner; permissive, and it never panics | +| `ast` | the typed syntax tree: declarations, lines, operands | +| `parser` | line-oriented parser producing the AST and its diagnostics | +| `arch` | register and instruction tables for the four architectures | +| `lint` | static checks over the AST | +| `format` | canonical formatter over the token stream | +| `lsp` | the language server | +| `asm` | standalone assembler: encoders, image layout, object emitters | +| `verify` | JIT execution, ABI checks, differential fuzzing | +| `debug` | interactive ptrace debugger | +| `cmd/gasm` | the CLI | +| `_gen` | rebuilds the `arch` tables from the Go toolchain source | + +The boundaries matter as much as the responsibilities: `ast` records syntax +only, and whether a name is a register or a label is left to `arch`, so the +parser stays architecture-agnostic. `asm` and `verify` are the only packages +that touch machine code and executable memory, and `cmd/gasm` owns no logic +beyond flags and output. ### `token` and `lexer` @@ -206,7 +233,7 @@ The standalone assembler (Phase 2). Its core is an amd64 instruction encoder: a REX/ModR-M/SIB/displacement/immediate engine plus the scalar instruction set, with the Plan 9 operand order (source first) mapped onto the x86 encoding. Every encoding is validated by decoding it again with `golang.org/x/arch`, the -one module dependency, used in tests only and never linked into the binary. +one module dependency, which also backs the `gasm dis` listings. A **RISC-V encoder** (Phase 5, RV64IMAFDC + RVC compression) encodes the full integer, atomic, float/double, FMA and CSR instruction sets with the MOV @@ -383,8 +410,8 @@ the Go ABI fixes across calls (amd64 `BP`/`R14`, arm64 `R29`/`R28`, riscv64 raw return trampoline `leaveJITCheckedRaw` verifies them, restoring the saved registers before Go code resumes. riscv64 is validated end to end under qemu-user emulation; arm64 shares the same stack convention and -fix; loong64 stays ground-truth-only until hardware validation (see -docs/DECISIONS.md). `gasm verify` runs the JIT checks when the host +fix; loong64 stays ground-truth-only until hardware validation. +`gasm verify` runs the JIT checks when the host matches the kernel's architecture and the toolchain comparisons elsewhere. @@ -436,14 +463,72 @@ watchdog is armed before the ptrace attach, so a sandboxed debuggee cannot block it), and `--cover` runs to completion with a breakpoint on every label and reports which blocks executed. -## Extension points +### Extending the toolkit - **New architecture:** add an entry to the generator in `_gen`, run `just gen`, and add a `buildXXX()` register file plus a case in `ForArch`. - **New lint rule:** add a function in `lint` and a rule-code constant. - **New LSP feature:** add a method case in `dispatch` and a handler. -The phases follow a dependency chain. Phase 1 (static analysis) builds only on -the AST; Phase 2 (the standalone assembler) emits object code; Phases 3 -(dynamic analysis) and 4 (the debugger) both consume the execution substrate -that the assembler provides. +## Data flow + +The main operation, assembling one file: + +```mermaid +sequenceDiagram + participant User + participant CLI as gasm CLI + participant Parser as parser + participant Asm as asm + participant Go as go toolchain + User->>CLI: gasm asm --format goobj -p pkg -o k.o k_amd64.s + CLI->>Parser: Parse(path, src) + Parser-->>CLI: AST, diagnostics + CLI->>Asm: AssembleFile(AST) + Asm->>Asm: encode operands, settle label offsets, lay out data + Asm-->>CLI: Image, code and data and relocations + CLI->>Asm: GOObject(pkg, path) + Asm->>Go: go list -json -export, externals only + Go-->>Asm: package and symbol indices + Asm-->>CLI: Go object bytes + CLI-->>User: wrote N bytes to k.o +``` + +Errors are produced where the parse or the encoding fails and become values at +the CLI boundary: the parser returns a diagnostic list and never aborts a file, +`AssembleFile` returns an error, and `cmd/gasm` prints what it has to stderr +and returns a non-zero exit code. The formatter and the linter take the same +AST by a different route: `gasm fmt` re-spaces the token stream and `gasm lint` +walks the parsed file, so neither depends on an encoding. + +## State and lifetime + +- The analysis packages (`lexer`, `parser`, `format`, `lint`, `arch`) hold only + read-only lookup tables and no mutable state: every call allocates its own + tokens and AST, and any number of goroutines may read the `arch` tables. +- A `verify.Kernel` owns one executable mapping, which `Close` releases. The + JIT trampolines keep the Go stack pointer and the checked-call sentinels in + package globals, so a call is a process-wide, one-at-a-time operation. The + `gasm verify` sweeps therefore run each function in a child process, which + contains a crash and keeps the globals unshared. +- `lsp.Server` is long-lived: it runs a single read and dispatch loop over the + stream and touches its document store only from that loop, so one server + serves one connection. +- A `debug.Session` owns a traced child process and pins its goroutine to the + forking OS thread, because ptrace requests must stay on that thread. + +## Dependencies + +- **`golang.org/x/arch`** (v0.30.0) is the one module dependency: it is the + disassembler backend (`gasm dis` and the debugger's listings) and the source + of the register metadata the encoder consults (`asm/reg.go`, `asm/vex.go`). + The tests additionally decode through it to validate the encodings. +- **The Go toolchain**, as an oracle and never as a library: `go tool asm` + supplies the object preamble and the ground truth for `gasm verify + --ground-truth`, `go list -json -export` locates the archives of the packages + a GOOBJ object references, and `_gen` parses + `$GOROOT/src/cmd/internal/obj//anames.go` to rebuild the tables. +- **Linux process interfaces** for the dynamic work: `mmap` and `mprotect` for + the JIT mapping, ptrace with `/proc/pid/mem` for the debugger. That is why + `verify` runs a JIT check only when the host architecture matches the + kernel's, and why `debug` is Linux-only. diff --git a/docs/CLI.md b/docs/CLI.md index b72aa16..af854df 100644 --- a/docs/CLI.md +++ b/docs/CLI.md @@ -1,215 +1,440 @@ -# CLI Reference +# Command line -Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit) +The reference below is taken from the program's own `--help`. If the two disagree, the +program is right and this file is a defect. -`gasm` is a single binary with subcommands. Run `gasm --help` for an -overview, or `gasm -h` for a command's usage and flags. +## Synopsis -## Global Flags +```sh +gasm [global flags] [command flags] [arguments] +``` -| Flag | Description | -|------|-------------| -| `-h`, `--help` | Show help | -| `-V`, `--version` | Print the version | +## Commands -## `gasm tokens ` +| Command | Purpose | +|---|---| +| `tokens` | print the lexical token stream | +| `parse` | parse a file and report syntax errors | +| `fmt` | canonicalise the formatting of `.s` files | +| `lint` | run the static checks | +| `asm` | assemble `.s` files to machine code | +| `dis` | disassemble machine code or an assembled file | +| `verify` | JIT-assemble and run the dynamic checks | +| `debug` | interactive source-level debugger | +| `diff` | compare the machine code of two `.s` files | +| `profile` | show the basic-block structure of the functions | +| `audit-instructions` | diff the encoder against the toolchain's name table | +| `scaffold` | generate a differential test skeleton for a kernel | +| `lsp` | run the language server over stdio | +| `version` | print the version | -Print the lexical token stream of FILE: position, token kind, and text, -one token per line. FILE may be `-` to read standard input. +## tokens -## `gasm parse ` +```text +Usage: gasm tokens +``` -Parse FILE and report syntax errors on stderr. On success, prints how -many declarations and TEXT functions the file contains. +Print the lexical token stream of FILE: position, token kind and text, one +token per line. FILE may be `-` to read standard input. -## `gasm fmt [-w|-l|-d] [path...]` +```sh +gasm tokens hello_amd64.s +``` -Canonicalise the formatting of Plan 9 assembly sources: indentation, -operand spacing, per-function mnemonic alignment, and blank-line layout. +```text +1:1 # "#" +1:2 IDENT "include" +1:10 STRING "\"textflag.h\"" +``` -| Flag | Description | -|------|-------------| -| `-w` | Write result to the source file (default: print to stdout) | -| `-l` | List files whose formatting differs, one per line; write nothing | -| `-d` | Print a unified diff of the canonical formatting instead | +## parse -With no arguments, or with a directory argument, every `.s` file below -it is reformatted in place and the names of changed files are listed -(`go fmt` style). `.` and `_` directories are skipped. +```text +Usage: gasm parse +``` -## `gasm lint ` +Parse FILE and report syntax errors on stderr. On success, print how many +declarations and TEXT functions the file contains. -Run static checks and print diagnostics as -`file:line:col: severity: message [code]`. Exit status is non-zero when -an error-severity diagnostic is found. +```sh +gasm parse hello_amd64.s +``` -| Flag | Description | -|------|-------------| -| `-disable` | Comma-separated rule codes to disable | +```text +hello_amd64.s: OK, 2 declarations, 1 functions +``` + +## fmt + +```text +Usage: gasm fmt [-w|-l|-d] [path...] +``` + +| Flag | Default | Effect | +|---|---|---| +| `-w` | off | write the result back to the source file | +| `-l` | off | list the files whose formatting differs; write nothing | +| `-d` | off | print a unified diff of the canonical formatting instead | + +`-l` and `-d` are mutually exclusive. With no arguments, or with a directory +argument, every `.s` file below it is reformatted in place and the names of the +changed files are listed, the way `go fmt` does; `.` and `_` directories are +skipped. Explicit file arguments print to stdout unless `-w` is given. + +```sh +gasm fmt -l kernel_amd64.s +``` + +Empty output means every file is formatted, which is the shape a CI check +wants; `-d` shows what would change: + +```sh +gasm fmt -d ugly_amd64.s +``` + +```text +--- ugly_amd64.s ++++ ugly_amd64.s +@@ -2,8 +2,8 @@ + + // func add(a, b int) int + TEXT ·add(SB), NOSPLIT, $0-24 +- MOVQ a+0(FP), AX +- ADDQ b+8(FP), AX ++ MOVQ a+0(FP), AX ++ ADDQ b+8(FP), AX +``` + +## lint + +```text +Usage: gasm lint +``` + +| Flag | Default | Effect | +|---|---|---| +| `-disable` | empty | comma-separated rule codes to disable | + +Diagnostics are printed as `file:line:col: severity: message [code]`. The exit +status is non-zero when an error-severity diagnostic is found; warnings (the +register-clobber audit, for example) do not affect it. Rules: `unknown-instruction`, `operand-count`, `undefined-label`, `duplicate-label`, `missing-ret`, `missing-textflag-include`, `abi-argsize`, `unreachable-code`, `register-clobber`, `funcdata-pcdata`, `unused-label`, `invalid-textflag`, `stack-imbalance`, `register-width-mismatch`, `abi0-register-args`, -`nonportable-register-name` and `unencodable-instruction`. +`nonportable-register-name`, `unencodable-instruction` and +`reserved-register-write`. -## `gasm asm [--format raw|elf|goobj] [-p pkg] [-o out] ` +```sh +gasm lint kernel_amd64.s +``` -Assemble FILE to machine code (amd64, arm64, riscv64, loong64). +## asm -| Flag | Description | -|------|-------------| -| `--format` | Output format: `raw` (default), `elf`, `goobj` | -| `-p` | Package path (required for `--format goobj`) | -| `-o` | Write output to file (default: hex dump to stdout) | +```text +Usage: gasm asm [--format raw|elf|goobj] [-p pkg] [-o out] +``` -## `gasm dis [-a arch] ` +| Flag | Default | Effect | +|---|---|---| +| `-format` | `raw` | output format: `raw` (concatenated image), `elf` or `goobj` (Go object) | +| `-p` | empty | package path for `--format goobj`, qualifying the exported symbols | +| `-o` | empty | write the output to this file instead of a hex dump on stdout | -Disassemble machine code to instruction text (via `golang.org/x/arch`). +Supported architectures: amd64 (VEX/AVX2 and EVEX/AVX-512 included), arm64, +riscv64 (RV64IMAFDC and RVC) and loong64, selected from the file's `_arch.s` +suffix. `raw` concatenates the functions and the data section into one +self-consistent image; `elf` emits a relocatable object that links with the +system toolchain; `goobj` emits the Go toolchain's own object format, which +`cmd/link` consumes directly. -With a `.s` file, the file is assembled first and the listing follows the -real layout: one block per `TEXT` function, local labels printed at their -offsets. The architecture comes from the file name suffix, or from `-a`. -With any other file, or `-` for standard input, the bytes are -disassembled linearly and `-a` selects the architecture (amd64, arm64, -riscv64 or loong64). +```sh +gasm asm hello_amd64.s +``` -| Flag | Description | -|------|-------------| -| `-a` | Architecture for raw input without a `_arch.s` name | +```text +add: 16 bytes + 0000: 48 8b 44 24 08 48 03 44 24 10 48 89 44 24 18 c3 +``` -## `gasm verify [flags] ` +## dis -Assemble FILE, map it into executable memory, and run dynamic checks. +```text +Usage: gasm dis [-a arch] +``` -| Flag | Description | -|------|-------------| -| `--ground-truth` | Compare machine code byte-for-byte against `go tool asm` | -| `--fuzz` | Differential fuzz: JIT both gasm and go-tool-asm, compare outputs | -| `-n` | Fuzz iterations per function (default: 1000) | -| `--abi` | Run ABI-checking calls (sentinel registers + red zone) | -| `--abi-n` | Number of ABI check iterations with varied inputs (default: 100) | -| `--profile` | List basic-block structure per function | -| `--smoke` | Call each NOSPLIT function with zeroed args | -| `--call ` | Invoke a single function with `--buf` instead of the sweeps | -| `--buf ` | Buffer spec for `--call`: `name:size:pattern[,name:size:pattern]` | -| `--args ` | Scalar args for `--call`: `name=value[,name=value]` (decimal or `0x` hex) | -| `--repeat ` | Number of times to repeat a `--call` invocation (default: 1) | -| `--save-corpus ` | With `--fuzz`: write each failing input to DIR as replayable JSON | -| `--replay ` | Re-run saved corpus entries (JSON in DIR), one child process per entry | +| Flag | Default | Effect | +|---|---|---| +| `-a` | empty | architecture for raw input without a `_arch.s` name | -The `--fuzz` mode runs each function in a subprocess; a partial function -(e.g. a decoder that faults on malformed input) is reported as -`CRASH` without killing the parent. Use `--call` with `--buf` to invoke -partial functions with valid data instead. +With a `.s` file the file is assembled first and the listing follows the real +layout: one block per `TEXT` function, local labels printed at their offsets. +With any other file, or `-` for standard input, the bytes are disassembled +linearly and `-a` selects the architecture (amd64, arm64, riscv64 or loong64). -The `--call` mode parses the `// func` signature, allocates the requested -buffers (`zero`, `ones`, `seq`, or a hex blob), builds the ABI0 argument -block with buffer pointers/lengths/capacities at the matching parameter -offsets, and prints the arg block before and after the call, showing -return values and any output written to the buffers. Scalar parameters -are supplied with `--args` (decimal, or `0x` hex) at their ABI0 offsets. +```sh +gasm dis hello_amd64.s +``` -The `--save-corpus` mode records the logical arguments (buffer contents and -scalars, not raw pointers) of every failing fuzz input as JSON. `--replay` -rebuilds a live argument block from each entry and calls it in its own child -process, reporting `OK`, `CRASH (reproduced)` or `FAIL` per entry and -exiting non-zero when any entry fails. +```text +add: 16 bytes + 0000: 48 8b 44 24 08 mov rax, qword ptr [rsp+0x8] + 0005: 48 03 44 24 10 add rax, qword ptr [rsp+0x10] + 000a: 48 89 44 24 18 mov qword ptr [rsp+0x18], rax + 000f: c3 ret +``` -## `gasm debug [--func ] [--buf spec] [--script file] ` +## verify -Interactive debugger for JIT-assembled functions (amd64, arm64, riscv64, -loong64). Requires a compiled binary on `$PATH` (not `go run`). +```text +Usage: gasm verify [-smoke] [-abi] [-fuzz] [-ground-truth] [-profile] [-call] +``` -| Flag | Description | -|------|-------------| -| `--func` | Function to debug (required) | -| `--buf` | Buffer spec: `name:size:pattern[,name:size:pattern]` | -| `--args ` | File containing the ABI0 argument block | -| `--script ` | Run REPL commands from a file (one per line) and exit; `-` reads stdin | -| `--timeout ` | Kill the debuggee after this duration (e.g. `30s`); for headless `--script` runs | -| `--cover` | Run to completion with a breakpoint on every instruction; report which executed, how often, and which labels were reached | +| Flag | Default | Effect | +|---|---|---| +| `--ground-truth` | off | compare the machine code byte-for-byte against `go tool asm` | +| `--fuzz` | off | differential fuzz against the `go tool asm` build | +| `-n` | 1000 | fuzz iterations per function | +| `--abi` | off | ABI-checking calls: sentinel registers and a red-zone canary | +| `--abi-n` | 100 | ABI check iterations with varied inputs | +| `--profile` | off | list the basic-block structure per function | +| `--smoke` | off | call each NOSPLIT function with zeroed arguments | +| `--call` | empty | invoke a single function with `--buf` instead of the sweeps | +| `--buf` | empty | buffer spec for `--call`: `name:size:pattern[,name:size:pattern]` | +| `--args` | empty | scalar args for `--call`: `name=value[,name=value]` (decimal or `0x` hex) | +| `--repeat` | 1 | number of times to repeat a `--call` invocation | +| `--save-corpus` | empty | with `--fuzz`: write each failing input to this directory as replayable JSON | +| `--replay` | empty | re-run saved corpus entries, one child process per entry | + +The JIT checks run when the host matches the file's architecture; the +toolchain comparison works everywhere. `--fuzz`, `--smoke` and `--abi` run each +function in its own child process, so a partial function that faults on random +input is reported as `CRASH` instead of ending the sweep; `--call` with `--buf` +invokes such a function with valid data. loong64 stays on the ground-truth path +until hardware validation. + +```sh +gasm verify --ground-truth hello_amd64.s +``` + +```text +hello_amd64.s: 1 functions JIT-loaded + add: MATCH (16 bytes) +ground truth: 1/1 functions byte-identical + add: 16 bytes, args=24, frame=0 NOSPLIT +``` + +```sh +gasm verify --call add --args a=2,b=3 hello_amd64.s +``` + +```text +add: 16 bytes, args=24 + signature: func add(a int, b int) int + scalars: + a = 2 + b = 3 + args before: 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (24 bytes) + args after: 02 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 (24 bytes) + call 1: OK +``` + +## debug + +```text +Usage: gasm debug --func +``` + +| Flag | Default | Effect | +|---|---|---| +| `-func` | empty | the function to debug, required | +| `-buf` | empty | buffer spec: `name:size:pattern[,name:size:pattern]` (zero, ones, seq or hex) | +| `-args` | empty | file containing the ABI0 argument block | +| `-script` | empty | run REPL commands from a file, one per line, and exit; `-` reads stdin | +| `-timeout` | 0 | kill the debuggee after this duration, for headless `-script` runs | +| `-cover` | off | run to completion with a breakpoint on every instruction and report which executed | + +The debugger spawns the debuggee from the `gasm` binary on `$PATH`, so install +it first with `just install`; `go run` does not work for the traced child. +Requires Linux (ptrace) and all four architectures are supported. REPL commands: -| Command | Description | -|---------|-------------| -| `break [if ]` | Set a breakpoint, optionally conditional | -| `delete ` | Remove a breakpoint | -| `info break` | List all breakpoints | -| `step [n]`, `s` | Single-step n instructions | -| `next`, `n` | Step over CALL | -| `finish`, `fin` | Run until the function returns | -| `continue`, `c` | Run until breakpoint, watchpoint or exit | -| `disas [n]`, `u` | Disassemble n instructions at PC | -| `regs` | Print general-purpose + vector/FP registers | -| `where` | Show source line and nearest label at PC | -| `stack` | Show stack near RSP (return address + ABI0 args) | -| `bt`, `backtrace` | Backtrace (current frame + return address) | -| `x [addr] [len]` | Hex-dump memory | -| `w ` | Write bytes to memory | -| `set ` | Set a register | -| `watch [r\|w] [size]` | Set a hardware watchpoint (write by default) | -| `unwatch []` | Clear one or all watchpoints | -| `labels`, `l` | List function labels and offsets | -| `help`, `h`, `?` | Show command help | -| `quit`, `q` | Kill the debuggee and exit | +| Command | Effect | +|---|---| +| `break [if ]` | set a breakpoint, optionally conditional | +| `delete ` | remove a breakpoint | +| `info break` | list the breakpoints | +| `step [n]`, `s` | single-step n instructions | +| `next`, `n` | step over a CALL | +| `finish`, `fin` | run until the function returns | +| `continue`, `c` | run until a breakpoint, watchpoint or exit | +| `disas [n]`, `u` | disassemble n instructions at the PC | +| `regs` | print the general-purpose and vector/FP registers | +| `where` | show the source line and the nearest label at the PC | +| `stack` | show the stack near RSP, the return address and the ABI0 args | +| `bt`, `backtrace` | backtrace: the current frame and the return address | +| `x [addr] [len]` | hex-dump memory | +| `w ` | write bytes to memory | +| `set ` | set a register | +| `watch [r\|w] [size]` | set a hardware watchpoint, write by default | +| `unwatch []` | clear one watchpoint or all of them | +| `labels`, `l` | list the function's labels and offsets | +| `help`, `h`, `?` | show the command help | +| `quit`, `q` | kill the debuggee and exit | -## `gasm diff [--map old=new,...] ` +```sh +gasm debug --func add --cover hello_amd64.s +``` -Compare the machine code produced by assembling two files. Shows which -functions differ and the first few differing bytes. Useful for verifying -that two implementations produce identical code, or for tracking encoding -changes between Go assembler versions. +## diff -| Flag | Description | -|------|-------------| -| `--map` | Comma-separated `old=new` pairs to match functions with different names | +```text +Usage: gasm diff +``` -Without `--map`, functions are paired by exact name. With `--map`, a -function named `old` in the first file is compared against the function -named `new` in the second file (e.g. `--map wideCopyAVX2=wideCopyAVX512` -pairs AVX2 and AVX-512 variants regardless of suffix). +| Flag | Default | Effect | +|---|---|---| +| `-map` | empty | comma-separated `old=new` pairs to match functions with different names | -## `gasm profile ` +Functions are paired by exact name unless `--map` says otherwise, so +`--map wideCopyAVX2=wideCopyAVX512` pairs two variants regardless of suffix. +The exit status is non-zero when anything differs. -Show the basic-block structure of functions in an assembly file. Lists -each function's labels, their offsets, and the block boundaries. This is -the static structure; for runtime execution counts, use `gasm verify ---fuzz` which exercises the code paths. +```sh +gasm diff hello_amd64.s hello_amd64.s +``` -## `gasm audit-instructions [amd64|arm64|riscv64|loong64]` +```text +add: identical (16 bytes) +all functions identical +``` -Compare the gasm encoder for the given architecture (default amd64) -against the installed `go tool asm` and print the diff: superset -encodings (gasm-only spellings, shippable via `gasm asm --format goobj`), -known-but-unencodable names (the encoder backlog) and go-only names -(feature gaps). The Go side is probed black-box with a battery of operand -shapes per mnemonic, so the audit tracks whatever toolchain -`go env GOROOT` provides. On non-amd64 architectures the backlog is an -over-approximation: a name counts as encodable only when a probe shape -assembles cleanly, so a name whose real forms the battery misses lands -in the backlog. +## profile -## `gasm scaffold differential ` +```text +Usage: gasm profile +``` -Print a differential test skeleton for every `// func` signature in -FILE. The generated test seeds random states, drives the kernel and a -portable reference (`Portable`), and compares outputs -byte-for-byte. Write the reference bodies, place the file in the -kernel's package, and run it in CI. +Show the basic-block structure of each function: its labels, their offsets and +the block boundaries. This is the static structure; for runtime execution +counts use `gasm debug --cover`, and for input coverage `gasm verify --fuzz`. -## `gasm lsp` +```sh +gasm profile hello_amd64.s +``` -Run the language server over standard input/output (JSON-RPC 2.0 with -Content-Length framing). Point an LSP-capable editor at the binary and -associate it with `.s` files. The target architecture is inferred from -the file-name suffix (`_amd64.s`, `_arm64.s`, `_riscv64.s`, -`_loong64.s`). +```text +add: 16 bytes, args=24, frame=0 NOSPLIT + basic blocks: 1 +``` -Provides: completion, hover, document symbols, push and pull -diagnostics, semantic tokens, go-to-definition, find references, rename, -document formatting, inlay hints, code actions, signature help, document -highlights, workspace symbol search, #include document links, and -folding ranges for function bodies. +## audit-instructions + +```text +Usage: gasm audit-instructions [amd64|arm64|riscv64|loong64] +``` + +Compare the gasm encoder for the given architecture (default amd64) against the +installed `go tool asm` and print the diff: superset encodings (gasm-only +spellings, shippable via `gasm asm --format goobj`), known-but-unencodable +names (the encoder backlog) and go-only names (feature gaps). The Go side is +probed black-box with a battery of operand shapes per mnemonic, so the audit +tracks whatever toolchain `go env GOROOT` provides. On non-amd64 +architectures the backlog is an over-approximation: a name counts as encodable +only when a probe shape assembles cleanly, so a name whose real forms the +battery misses lands in the backlog. + +```sh +gasm audit-instructions amd64 +``` + +```text +gasm table (amd64, families excluded): 1542 mnemonics +gasm encodable: 580 go tool asm recognized: 1542 +shared: 580 +``` + +## scaffold + +```text +Usage: gasm scaffold differential +``` + +Print a differential test skeleton for every `// func` signature in FILE. The +generated test seeds random states, drives the kernel and a portable reference +(`Portable`), and compares the outputs byte-for-byte. Write the reference +bodies, place the file in the kernel's package, and run it in CI. + +```sh +gasm scaffold differential kernel_amd64.s > kernel_differential_test.go +``` + +## lsp + +```text +Usage: gasm lsp +``` + +Run the language server over standard input/output, JSON-RPC 2.0 with +`Content-Length` framing. Point an LSP-capable editor at the binary and +associate it with `.s` files; the target architecture is inferred from the +file-name suffix (`_amd64.s`, `_arm64.s`, `_riscv64.s`, `_loong64.s`). + +Provides: completion, hover, document symbols, push and pull diagnostics, +semantic tokens, go-to-definition, find references, rename, document +formatting, inlay hints, code actions, signature help, document highlights, +workspace symbol search, #include document links, and folding ranges for +function bodies. Definition, references and rename work across every open +document. + +## version + +```text +Usage: gasm version +``` + +Print the version the toolchain recorded for the build, the same string as +`gasm --version`: the tag on a tagged checkout, a pseudo-version naming the +commit below one, with `+dirty` appended on a dirty tree and `(devel)` outside +version control. + +## Global flags + +| Flag | Default | Effect | +|---|---|---| +| `-h`, `--help` | off | print the usage | +| `-V`, `--version` | off | print the version | + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | success | +| `1` | a failure the program detected: a parse or assembly error, an error-severity lint diagnostic, a mismatch in `verify`, a file that cannot be read | +| `2` | the arguments were wrong: a missing or extra argument, an unknown command or format, an invalid `--map` pair | + +## Examples + +Assemble a kernel, check it, and run it: + +```sh +gasm lint kernel_amd64.s +gasm fmt -l kernel_amd64.s +gasm asm -o kernel.bin kernel_amd64.s +gasm verify --ground-truth kernel_amd64.s +``` + +Link the kernel into a Go program through the toolchain's own object format: + +```sh +gasm asm --format goobj -p example.com/kernel -o kernel.o kernel_amd64.s +``` + +Find which labels a failing kernel reaches, headlessly: + +```sh +gasm debug --func decodeBlockAVX2 --cover --script cmds.txt --timeout 30s kernel_amd64.s +``` diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md deleted file mode 100644 index 47c8c60..0000000 --- a/docs/DECISIONS.md +++ /dev/null @@ -1,111 +0,0 @@ -# Deferred decisions - -Design decisions deliberately postponed, with enough context to pick them up -again without re-deriving the analysis. Each entry records what is deferred, -why, the options on the table, and the trigger that should reopen it. - ---- - -## GOOBJ external (cross-package) symbol references - -**Status:** resolved (v0.29.0+, 2026-08-07). - -**Approach taken.** Instead of parsing the compiler's iexport data (which -would have required either `golang.org/x/tools` or an in-house parser), the -resolver reads the **GOOBJ data directly** from the target package's `.a` -archive. The `.a` file contains a `_go_.o` member whose GOOBJ s is the -same one gasm writes; the parser reuses the same layout (`blkSymdef`, -`blkNonpkgdef`, the string table), so no new dependency was needed. - -**How it works.** - -1. `go list -json -export ` finds the target package's `.a` file. -2. `extractGOOBJ` reads the ar archive, finds the `_go_.o` member, skips - the `"go object …\n!\n"` preamble and parses the GOOBJ header. -3. `goobjFile.symbols()` walks `blkSymdef` and `blkNonpkgdef` in definition - order (the same order the linker uses) to build the symbol-to-index - mapping. -4. `resolveExternalSymbols` wires the resolved `{PkgIdx, SymIdx}` into the - GOOBJ emission. - -The resolver is invoked automatically when `img.Externals` is non-empty; it -runs `go list` as a subprocess (consistent with `toolchainObjectPreamble` -which already calls `go tool asm`). All symbol data is cached per package -for the lifetime of the GOOBJ emission. - -## 2026-08-30 non-amd64 JIT execution trampolines - -**Status:** resolved for riscv64 (validated end to end under qemu-user) -and arm64 (fix in place, consistent with the observed frame convention); -open for loong64 until hardware validation. - -**Root cause (found 2026-08-31).** The trampolines advanced SP past the -leave-address slot after loading it, while the assembled kernels read -their first argument at SP+8 per the frame convention (the amd64 path -already kept SP on that slot). Removing the advance fixed riscv64 -immediately (plain and checked ABI tests pass under qemu-user); the -arm64 kernel's pre-fix trace showed exactly the same SP+8 reading. The -apparent arm64/loong64 "crashes in the JIT" turned out to be dominated -by an unrelated instability: the Go 1.26 and 1.27 runtimes crash under -qemu-user arm64 emulation (GC worker start, identical signature with the -JIT tests skipped, both qemu 7.2 and 10.2), and the Go loong64 runtime -does not start at all. `gasm verify` therefore keeps loong64 kernels on -the ground-truth path until hardware validation; the GOARCH-guarded -tests (`verify/jit_arch_test.go`, `verify/abi_arch_test.go`) are the -hardware validation entry point. - -**State.** The per-architecture trampolines compile for all targets, the -kernels they execute are byte-for-byte correct against `go tool asm`, and -under `qemu-aarch64` the arm64 kernel demonstrably executes and stores its -result correctly. The failure is on the return path into Go code: arm64 -and loong64 take a SIGSEGV after the kernel's RET (the Go-side unwind -through `leaveJIT` and its interposed ABIInternal wrapper is the suspect), -and riscv64 returns cleanly but with an untouched result area. amd64 is -unaffected (the checked trampoline saves and restores BP/R14 and the flow -is validated end to end). - -**Evidence harness.** `verify/jit_arch_test.go` (plain call) and -`verify/abi_arch_test.go` (checked call) are GOARCH-guarded tests; build -the test binary per target (`GOARCH=arm64 go test -c -o v.test ./verify/`) -and run it under `qemu-aarch64-static` from the `verify/` directory. A -minimal reproducer pattern lives in the qemu exploration notes: verify -loads, the kernel executes, the fault follows the return. - -**Fix direction.** Compare the amd64 checked trampoline (GLOBL/DATA raw -address, explicit SP/BP/R14 save-restore) against the arm64/riscv64/ -loong64 `leaveJIT` unwind, in particular the interaction with the -ABIInternal wrapper that `reflect.ValueOf(leaveJIT).Pointer()` returns. -The plain-call path (no sentinels) fails the same way, so the checked -path is not the variable. - ---- - -## 2026-08-29 tooling round - -- `lint abi0-register-args`: flags kernels whose `// func` parameters are - never read from the FP frame. Motivated by a real latent bug: kernels - reading arguments from registers pass every test while the autogenerated - `F.abi0` wrapper happens to leave the caller's register values intact, and - break on a toolchain upgrade. -- `lint nonportable-register-name`: the RAX/EAX register spellings are a gasm - extension; go tool asm rejects them, so files using them only link through - the gasm goobj path. -- `lint unencodable-instruction`: a mnemonic in the architecture table that - `asm.Encodable` rejects is flagged at edit time instead of failing at - assembly time. -- `audit-instructions`: black-box diff of the encoder against go tool asm. - As of this round the tables fully overlap on names; the audit exists to - catch drift in both directions (future supersets and future gaps). -- `scaffold differential`: generates the direct-call differential skeleton - (two independent seed sets, output and in-place buffer comparison) that a - pipeline-level fuzz can never replace. -- `verify --args`: scalar arguments for `-call`, closing the repro gap where - only buffers could be supplied. -- `debug --script/--timeout/--cover`: headless debugging with a watchdog - armed before the ptrace attach (untracing sandboxes hang the attach), and - label-level block coverage for the "did my test ever enter that branch" - question. -- Superset policy remains: gasm may accept spellings and encodings go tool - asm lacks, but such kernels ship only via `gasm asm --format goobj`; the - audit reports the superset surface. The register-alias superset is warned - about by lint because the default `go build` path cannot consume it. diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 9ecfbab..93adf49 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -4,11 +4,13 @@ Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrb ## Prerequisites -- **Go** 1.27+ with `toolchain go1.27.0` +- **Go** 1.27.1, the exact version the `go` directive in `go.mod` declares - **just**, the command runner; every task below is a just recipe +- A Linux host on amd64, arm64, riscv64 or loong64: `gasm debug` needs ptrace + and the JIT checks of `gasm verify` need executable memory - No external dependencies beyond the Go toolchain -## Quick Start +## Setup ```sh git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git @@ -17,105 +19,65 @@ just build # compile bin/gasm, zero errors and zero warnings just gates # build, fmt-check, vet, test, race: the definition of done ``` -## Just Recipes +## Recipes -### `just build` +Every recipe in the `justfile`, and what it does. -Compiles `bin/gasm` with `CGO_ENABLED=0` and stripped symbols. Zero -errors, zero warnings. This is the minimum bar before any commit. - -### `just gates` - -The definition of done, in one command: `build`, `fmt-check`, `vet`, -`test` and `race`, in that order. Run it once per task, never per edit; -`just unit` is the one that runs after every edit. +| Recipe | What it does | +|---|---| +| `just build` | compiles `bin/gasm` with `CGO_ENABLED=0` and stripped symbols; zero errors and zero warnings | +| `just test` | the test gate: the suite with `-count=1`, the coverage profile and the 80 % floor | +| `just race` | the same suite under the race detector; the expensive one, so it runs once, inside `gates` | +| `just unit [packages] [run]` | fast, cached, scoped run for iterating: no race and no coverage, so an unchanged package reports instantly | +| `just fuzz [fuzztime]` | time-boxed fuzz of one target; the package is required, because `go test -fuzz` refuses more than one | +| `just bench [packages]` | benchmarks (`-benchmem -count=5`); on an idle machine only | +| `just fmt` | formats the tree in place with `gofmt` | +| `just fmt-check` | zero diff; prints nothing when everything is formatted, which is the shape the CI step wants | +| `just vet` | both static gates: `go vet` and `go fix -diff` | +| `just gates` | `build`, `fmt-check`, `vet`, `test` and `race`, in that order: the definition of done | +| `just clean` | removes the build artefacts, `bin/` and `coverage.out` | +| `just install` | builds, then copies the binary into `bindir` (`~/.local/bin`); `gasm debug` needs an installed binary, because it spawns the debuggee from `$PATH` | +| `just uninstall` | removes the installed binary from `bindir` | +| `just run` | runs the CLI with `go run -buildvcs=true`; the recipe takes no arguments, so flags go through the package instead | +| `just dev` | the same as `run`; the project has no watcher to add | +| `just gen` | regenerates the `arch` instruction tables from the Go toolchain source; not a gate | ### `just test` ```sh -go test -count=1 -timeout 30m -coverprofile=coverage.out \ - -coverpkg=./arch/...,./asm/...,./ast/...,./disasm/...,./format/...,./lexer/...,./lint/...,./lsp/...,./parser/...,./token/...,./verify/... \ - ./... +go test -count=1 -timeout 10m -coverprofile=coverage.out \ + ./arch/... ./asm/... ./ast/... ./disasm/... ./format/... ./lexer/... \ + ./lint/... ./lsp/... ./parser/... ./token/... ./verify/... ``` -The whole suite runs (`-count=1`, so no cached pass counts), which keeps -the packages that need hardware and the CLI glue under the gate. The -coverage floor is computed over the product packages only (arch, asm, -ast, disasm, format, lexer, lint, lsp, parser, token, verify; `debug` -traces a live process and `cmd/gasm` is CLI glue) and fails if the total -is below 80 %. +The suite runs over the logic packages (`-count=1`, so no cached pass +counts): arch, asm, ast, disasm, format, lexer, lint, lsp, parser, +token, verify. `debug` traces a live process and `cmd/gasm` is thin CLI +glue, so both sit outside the sweep, and a thin `cmd/` in it would drag +the coverage total under the floor. The floor fails if the total is +below 80 %. CI runs the same command with the same ten-minute bound, so +the number is the same everywhere. -### `just race` - -The same suite under the race detector. The expensive one, so it runs -once, inside `gates`. - -### `just unit` +### `just run` ```sh -just unit ./asm/ TestVexGroundTruth +just run +go run -buildvcs=true ./cmd/gasm lint kernel_amd64.s +go run -buildvcs=true ./cmd/gasm verify --ground-truth kernel_amd64.s ``` -Fast, cached, scoped run for iterating: no race, no coverage, so an -unchanged package reports instantly. - -### `just fuzz ` - -Time-boxed fuzz of one target in one package; the package is required, -because `go test -fuzz` refuses more than one. Seed corpora run as plain -tests in `unit` and `test`. Never a gate. - -### `just bench` - -Benchmarks (`-benchmem -count=5`). On an idle machine only. - -### `just fmt`, `just fmt-check`, `just vet` - -`fmt` formats in place. `fmt-check` prints nothing when everything is -formatted, which is the shape the CI step wants. `vet` runs both static -gates: `go vet` and `go fix -diff`. - -### `just run -- ` - -Runs the CLI via `go run -buildvcs=true`: - -```sh -just run -- lint kernel_amd64.s -just run -- fmt -w kernel_amd64.s -just run -- verify --ground-truth kernel_amd64.s -``` - -### `just install` - -Builds and copies the `gasm` binary into `~/.local/bin` (`BINDIR` -overrides the destination). - -### `just uninstall`, `just clean` - -`uninstall` removes the installed binary from `bindir`. `clean` removes -the build artefacts: `bin/` and `coverage.out`. - -## Version reporting - -The version is never injected. `gasm --version` prints what the -toolchain recorded in the build information: the tag on a tagged -checkout, a pseudo-version naming the commit below one, `+dirty` on a -dirty tree, and `(devel)` outside version control. There is no -`-ldflags "-X"` anywhere and no version constant in the source. +The flag on `go run` is there because it does not stamp the build otherwise, +which `--version` would then report as `(devel)`. ### `just gen` -Regenerates the architecture instruction tables in `arch/` by parsing -the Go toolchain's own assembler source +Regenerates the architecture instruction tables in `arch/` by parsing the Go +toolchain's own assembler source (`$GOROOT/src/cmd/internal/obj//anames.go`). Requires a Go installation. Output is committed, with no runtime dependency on the toolchain. -### `just uninstall` - -Removes `coverage.out`, the `gasm` binary, and `*.test` artefacts. - -## Running Individual Tests +## Running a single test ```sh go test -run TestVexGroundTruth ./asm/ @@ -124,32 +86,53 @@ go test -run TestGOObjectLinkAndRun ./asm/ go test -run TestFuzzWideCopy ./verify/ ``` -## Debugger Note +Add `-v` for the sub-test names, and `-race` when the change touches +concurrency. `-count=1` defeats the test cache when a result looks stale. -`gasm debug` spawns a child process from the binary on `$PATH`. It does -not work with `go run`; install first: +## Coverage ```sh -just install -gasm debug --func decodeBlockAVX2 path/to/kernel_amd64.s +just test +go tool cover -func=coverage.out ``` -## Project Layout +The `total:` line is the number that matters, and it stays at 80 percent or +more. +## Debugging the build + +```sh +go build -gcflags='-m' ./... # inlining decisions +go build -gcflags='-S' ./... # what the compiler generated +go tool asm -S kernel_amd64.s # how the toolchain's assembler encodes a kernel +gasm dis kernel_amd64.s # what gasm makes of the same kernel +gasm tokens kernel_amd64.s # the token stream +gasm profile kernel_amd64.s # the basic blocks of each function ``` -cmd/gasm/ CLI entry point (subcommands) -token/ Lexical token kinds and positions -lexer/ Hand-written scanner -ast/ Abstract syntax tree -parser/ Line-oriented parser -arch/ Register and instruction tables (generated) -lint/ Static analysis rules -format/ Canonical formatter -lsp/ Language Server Protocol server -asm/ Standalone assembler, encoder, object emitters -verify/ JIT execution, differential testing, ABI checks -debug/ Interactive ptrace debugger (all four architectures) -_gen/ Instruction table generator -testdata/ Test fixtures -docs/ Architecture, development, CLI reference -``` + +`gasm verify --ground-truth` is the differential check that ties the two +together: it compares gasm's bytes with `go tool asm`'s, with the relocation +sites masked, so an encoding drift shows up as a byte difference rather than a +crash later. + +## Continuous integration + +Workflows live in `.gitea/workflows/` and run on the project's own runners: +Test on a push or pull request to `development`, race dispatched by hand, and +the release on a `v*` tag. They are written by hand rather than through +`just`, but they enforce the same set of gates minus the race detector, which +the shared runner cannot afford on a push; a green `just gates` locally is +therefore the fastest way to a green pipeline. + +## Releases + +Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`, +which triggers the release workflow: it builds the portable Linux targets, +takes the notes from the matching `CHANGELOG.md` section and uploads the +assets. + +The version is never injected. `gasm --version` prints what the +toolchain recorded in the build information: the tag on a tagged +checkout, a pseudo-version naming the commit below one, `+dirty` on a +dirty tree, and `(devel)` outside version control. There is no +`-ldflags "-X"` anywhere and no version constant in the source.