docs: add the full documentation surface — AGENTS, CONTRIBUTING, cli and development references

Assisted-by: DeepSeek V4 Flash
This commit is contained in:
2026-08-01 05:22:00 +02:00
parent 89d633f4bb
commit c77d68018c
7 changed files with 1519 additions and 0 deletions
+119
View File
@@ -0,0 +1,119 @@
# AGENTS.md — gasm-devkit
Repository rules for AI agents and contributors. Read before modifying any
code in this repository.
## AI Contribution Policy
AI agents may assist with code, documentation, tests, and review in this
repository. All AI-assisted changes must:
- Follow the code style and conventions in this file.
- Include the trailer `Assisted-by: <model-name>` in every commit message.
- Not commit directly to `main` — work on `development`.
- Pass the full Definition of Done before any commit.
## Workflow
- **Branching.** `development` is the working branch. `main` is
release-only: merge from `development`, then tag. Never commit directly
to `main`.
- **Release procedure.**
1. Bump `version` in `justfile` and `cmd/gasm/main.go`.
2. Update `CHANGELOG.md` with a new `## [X.Y.Z] — YYYY-MM-DD` section.
3. Update `README.md` and `docs/ARCHITECTURE.md` if user-visible
behaviour changed.
4. Run the Definition of Done (below).
5. Commit on `development`.
6. `git checkout main && git merge --ff-only development`.
7. `git tag vX.Y.Z`.
8. `git checkout development`.
9. `GOBIN=~/.local/bin just install-bin`.
## Commit Messages
Conventional Commits, subject line only, imperative mood, lowercase after
the colon:
```
feat(asm): add EVEX gather and scatter with VSIB addressing
```
Allowed types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`,
`chore`, `ci`, `build`, `revert`.
Every commit ends with exactly one trailer, using the model that
assisted with the change:
```
Assisted-by: <model-name>
```
Replace `<model-name>` with the actual model (e.g. `DeepSeek V4 Pro`).
No body, no footers, no trailing period on the subject.
## Code Style
Language: Go 1.26 (`toolchain go1.26.5`).
### Formatter
`gofmt` — zero diff. Run `just fmt` before committing.
### Linter
`go vet` — zero warnings. Run `just build` before committing.
### Tests
`go test -race -count=1 ./...` — all green, coverage ≥ 80 % (hard gate,
enforced by `just test`).
### Dependencies
- **Production code:** standard library only. No third-party imports in
shipped code.
- **Test code:** `golang.org/x/arch` is the sole test dependency (decode
oracle for round-trip validation). It is never linked into the binary.
- **No cgo, no C, no external toolchains, no JavaScript.**
### Error Handling
Explicit `if err != nil`. Wrap with `fmt.Errorf("context: %w", err)`.
No panics outside `main`. The one exception: the JIT trampoline's
`recover`-guarded decoder hot path, which converts bounds panics to
sentinel errors.
### Assembly
Plan 9 syntax (Go's assembler dialect). Hand-written — no code generators
except `_gen/gen.go` for instruction tables (which parses the Go
toolchain source). Every instruction table is committed; no runtime
dependency on the Go toolchain.
### File Naming
- `_amd64.s`, `_arm64.s`, `_riscv64.s`, `_loong64.s` for
architecture-specific assembly.
- `_linux_amd64.go` for platform-specific Go files.
- `_test.go` suffix for test files.
## Definition of Done
A task is not complete until all of these pass:
1. `just build` — `go vet` + `gofmt` check, zero errors, zero warnings.
2. `just test` — full suite with `-race`, coverage ≥ 80 %.
3. `just fmt` — produces no diff.
4. Diagnostics — zero warnings across the project.
5. Non-trivial changes reviewed.
## Licence
BSD-3-Clause. Every source file carries the SPDX header:
```
// Copyright (c) 2026 Petr Balvín <opensource@petrbalvin.org> (https://petrbalvin.org)
// SPDX-License-Identifier: BSD-3-Clause
```
+722
View File
@@ -0,0 +1,722 @@
# Changelog
All notable changes to gasm-devkit are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Conventional Commits](https://www.conventionalcommits.org/).
## [development]
Unreleased changes on the `development` branch.
## [0.27.0] — 2026-08-01
Subprocess isolation for `--fuzz`: each function is fuzzed in its own child
process, so a partial function (decoder) that faults on random garbage is
reported as "CRASH (partial function, use --ground-truth)" without killing
the parent. CRASH is informational (exit 0); only MISMATCH is an error.
### Fixed
- `gasm verify --fuzz` no longer crashes the process on partial functions.
## [0.26.0] — 2026-07-31
Universal differential fuzzing: `gasm verify --fuzz` needs no hand-written
reference. It parses the `// func` signature from the assembly source,
generates typed random inputs (slices with random content, ints, pointers to
fixed arrays), JIT-executes BOTH the gasm-assembled and the go-tool-asm-
assembled versions with independent buffer copies, and compares the result
area bit-for-bit.
### Added
- `verify`: `FuzzFunc` / `ExtractSignatures` / `parseFuncSig` — universal
differential fuzz driven by the conventional `// func` comment. Each
version gets its own buffer set (deep copy) so functions that write to
their arguments (histogram increments) don't corrupt the other's input.
- `gasm verify --fuzz [-n N]`: runs the differential fuzz for every function
with a parseable signature. Total functions (wideCopy, pack16, decorrelate,
analyze, autocorr) pass; partial functions (decoders that fault on malformed
input) should use `--ground-truth` instead.
### Known limitation
`--fuzz` crashes the process for partial functions (e.g. LZ4 decoders) whose
over-copy paths read past the buffer on random garbage input. Subprocess
isolation (fork per function) is planned. Use `--ground-truth` for decoders.
## [0.25.0] — 2026-07-30
Universal ground-truth verification: `gasm verify --ground-truth` assembles
any `.s` file with both gasm and `go tool asm`, then compares the machine
code byte-for-byte per function (relocation sites masked). No hand-written
reference needed — the Go toolchain IS the oracle.
### Added
- `verify`: `GroundTruth` — shells out to `go tool asm`, parses the GOOBJ
output (minimal reader: block offsets, nonpkg symbol table, data index)
and returns per-function code bytes.
- `gasm verify --ground-truth`: compares gasm's output against the Go
assembler's, reporting MATCH/MISMATCH per function with the first
differing byte. Relocation disp32 fields (static-symbol references the
linker fills) are masked before comparison.
- Verified: go-lz4 AVX2 2/2, go-flac AVX2 17/17 functions byte-identical.
## [0.24.0] — 2026-07-29
The full analyze family and stereo PCM decode are now differentially tested.
15 of 17 go-flac AVX2 kernels have bit-for-bit differential coverage; the
two remaining (autocorrAVX2 — FMA reassociation, lpcResidualAVX2 — complex
multi-arg) are deferred.
### Added
- `verify`: `analyzeO3RangeAVX2` and `analyzeO4RangeAVX2` differential tests
(200 iterations each, same harness as O1/O2/Res).
- `verify`: `decodeStereo16AVX2` differential test (500 random interleaved
stereo PCM buffers, both channels compared sample-by-sample).
## [0.23.0] — 2026-07-28
The analyze family and 24-bit PCM decode join the differential suite.
### Added
- `verify`: `analyzeO2RangeAVX2` and `analyzeResRangeAVX2` differential
tests (200 iterations each, shared harness with O1: zigzag fold, partial
sum, overflow flag and Len32 histogram).
- `verify`: `decodeMono24AVX2` differential test (500 random 24-bit PCM
buffers, sign-extension compared sample-by-sample).
## [0.22.0] — 2026-07-27
The remaining go-flac encoder kernels join the differential suite.
### Added
- `verify`: `analyzeO1RangeAVX2` differential test (300 random partitions:
zigzag fold, partial sum, overflow flag and the 32-bin Len32 histogram
compared element-by-element against the portable Go reference).
- `verify`: `fastStereoSumsAVX2` differential test (300 random stereo
frames: the four zigzag-fold entropy sums compared against the scalar
loop).
### Verified
- `gasm fmt` doc-comment indentation confirmed correct: comments before
every TEXT are at column 0 (the RET-detection logic handles multi-exit
functions).
## [0.21.0] — 2026-07-26
Differential testing extended to all four production kernels and the CLI
exposes the full dynamic-analysis toolkit.
### Added
- `verify`: go-flac AVX2 differential tests — `decodeMono16AVX2` (500
random PCM buffers), `pack16AVX2` (500 random int32→int16 packings) and
all four decorrelation kernels (200 iterations each: left-side, side-right,
mid-side, interleave) compared bit-for-bit against the portable Go
references.
- `verify`: go-lz4 AVX-512 differential tests — `decodeBlockAVX512` (3 000
fuzzed LZ4 blocks + known answers) and `wideCopyAVX512` (0–1024 bytes)
against the same portable oracle as the AVX2 suite.
- `gasm verify --abi`: runs each NOSPLIT function with sentinel registers
and a red-zone canary, reporting violations.
- `gasm verify --profile`: lists the static basic-block count per function.
## [0.20.0] — 2026-07-25
Coverage profiling: the third pillar of Phase 3. Static basic-block
enumeration from the assembler's label map, combined with multi-input path
diversity measurement — how many observationally distinct execution paths a
test corpus exercises.
### Added
- `verify`: `Kernel.Blocks` / `Kernel.BlockCount` — enumerate basic blocks
from the assembler's local-label map (every jump target is a block
boundary; the function entry is always a block). `decodeBlockAVX2` has
27 blocks.
- `verify`: `Kernel.ProfilePaths` — run the function with a corpus of
argument blocks and collect distinct output fingerprints (the result
words); reports path diversity as a lower bound on code coverage.
### Note
INT3-based per-block hit counting was prototyped but deferred: Go's runtime
signal management (sigaltstack, handler re-installation) makes raw
rt_sigaction handlers fragile in a Go process. The static + path-diversity
approach delivers the project's goal (proving the SIMD path and tail handling
execute) without fighting the runtime.
## [0.19.0] — 2026-07-24
Runtime ABI checks: the second pillar of Phase 3. The JIT trampoline now
has an ABI-checking variant that sets sentinels in the callee-saved registers
(BP, R14) before entering the assembled function and verifies they survive on
return, plus a red-zone canary (128 bytes below SP filled with 0xA5) that
detects any illegal write below the stack pointer.
### Added
- `verify`: `CallChecked` / `Kernel.CallFuncChecked` — ABI-checking JIT call
with sentinel registers and red-zone canary; returns an `ABIReport`
(BPClobbered, R14Clobbered, RedZoneHit).
- `verify`: the raw `leaveJITCheckedRaw` trampoline — a TEXT symbol with no
ABIInternal wrapper (address obtained via GLOBL/DATA), so the JIT
function's RET lands directly in the check code and sees the registers
exactly as the function left them.
- Tests: deliberate BP/R14 clobberers detected; both go-lz4 kernels
confirmed ABI-clean (BP preserved, R14 preserved, red zone intact).
## [0.18.0] — 2026-07-23
Differential testing: the JIT-assembled go-lz4 `decodeBlockAVX2` kernel is
fuzzed against a portable Go reference — 5 000 valid LZ4 blocks compared
bit-for-bit, plus 2 000 hostile (random garbage) inputs with matching error
codes. This is the automated form of the project's bit-identical contract.
### Added
- `verify`: differential fuzz tests — a random LZ4 block generator produces
valid blocks (literals, overlapping matches, extension bytes) and the
JIT-assembled kernel's output is compared byte-for-byte against a portable
Go decoder; a hostile-input suite confirms error-code agreement on random
garbage (no crashes, same classification).
## [0.17.0] — 2026-07-22
Phase 3 begins: dynamic analysis. A JIT execution substrate that assembles
Plan 9 amd64 kernels into executable memory and calls them directly — pure Go
(stdlib only, `syscall.Mmap` + an assembly trampoline), no cgo, no external
toolchain.
### Added
- `verify` package: JIT infrastructure — `Map` copies machine code into a
W^X memory mapping, `Call` invokes it through an ABI0 trampoline that
switches to a prepared stack and back. `Load`/`LoadSource`/`LoadAST`
parse, assemble and map a `.s` file in one step; `Kernel.CallFunc`
marshals the argument block and returns results.
- `gasm verify` subcommand: assembles a file, JIT-loads it and reports the
available functions; with `-smoke`, calls each NOSPLIT function with
zeroed arguments to confirm the trampoline works end-to-end.
- Integration tests: the go-lz4 `decodeBlockAVX2` and `wideCopyAVX2`
kernels (699 and 146 bytes) assemble, map and execute correctly —
known-answer LZ4 blocks decode bit-for-bit, wide copies of 0–1024 bytes
match, malformed input returns the correct error codes.
### Verified
- `just test` (race, 84.6 % total coverage, verify 82.2 %).
- `gasm verify` on both go-lz4 kernels: all functions JIT-load and
smoke-test clean.
## [0.16.0] — 2026-07-21
The scalar conversions between vector and general-purpose registers — the
last of the amd64 EVEX instruction set.
### Added
- `asm`: the GPR-interchanging conversions, byte for byte against the Go
assembler (28 ground-truth cases including memory sources and extended
GPRs): vector to GPR — the signed and truncated VCVT{,T}S{D,S}2SI{,Q}
in both VEX and EVEX, and the unsigned VCVT{,T}S{D,S}2USI{L,Q}
(EVEX only); GPR to vector — VCVTSI2SD{L,Q}/VCVTSI2SS{L,Q} (VEX and
EVEX) and VCVTUSI2SD{L,Q}/VCVTUSI2SS{L,Q} (EVEX only), whose preserved
vector source sits in vvvv (three Plan 9 operands).
## [0.15.0] — 2026-07-20
The last of the EVEX conversions and narrowing/extending moves — the EVEX
instruction set is now complete save for the GPR-interchanging forms.
### Added
- `asm`: the unsigned and truncating conversions — VCVTPD2PS (and the X/Y
spellings, whose length the spelling fixes), VCVTPD2UDQ (X/Y),
VCVTTPD2UDQ (X/Y), VCVTTPD2UQQ, VCVTPS2UDQ, VCVTTPS2UDQ, VCVTPS2UQQ,
VCVTTPS2UQQ, VCVTTPD2QQ, VCVTTPS2QQ, VCVTUQQ2PD, VCVTUQQ2PS (X/Y) and
VCVTQQ2PS X/Y.
- `asm`: the remaining sign/zero-extending moves (VPMOVSXBD/BQ/WQ and
VPMOVZXBD/BQ/WD/WQ, VEX and EVEX) and the complete signed and unsigned
narrowing stores (VPMOVS{DB,QB,DW,QW,QD,WB}, VPMOVUS{DB,QB,DW,QW,QD,WB},
VPMOVDB, VPMOVQW).
- `asm`: the mask/vector conversions (VPMOVM2B/W/D/Q and VPMOVB2M/W2M/
D2M/Q2M), whose K register is a genuine operand rather than a mask and
which therefore take no masking suffixes.
## [0.14.0] — 2026-07-19
The floating-point helper and conversion tail of the AVX-512 set, plus
gather and scatter with VSIB addressing — every encoding verified byte for
byte against the Go assembler.
### Added
- `asm`: the floating-point helpers — reciprocals and reciprocal square
roots (VRCP14/VRSQRT14 PD/PS/SD/SS), exponents and mantissas (VGETEXP*,
VGETMANT*), scaling by powers of two (VSCALEF*), rounding (VRNDSCALE*),
reduction (VREDUCE*), immediate fixup (VFIXUPIMM*) and range selection
(VRANGE*), and floating-point class tests (VFPCLASSPD/PS X/Y/Z and
VFPCLASSSD/SS — a new immediate form whose reg field carries the opmask
destination).
- `asm`: **gather and scatter with VSIB addressing.** The gathers take
both Go spellings: the VEX form with a vector mask register (OP mask,
vsib, dst) and the EVEX form with an explicit K mask (OP vsib, K, dst),
where the EVEX L'L field follows the VSIB index register rather than the
data register (a ZMM index with an YMM destination encodes L'L = 10, as
the Go assembler emits). The scatters (VSCATTER*/VPSCATTER*) are EVEX
only (OP src, K, vsib). All eight gather and eight scatter widths.
- `asm`: the remaining conversions — VCVTQQ2PS (the 512-bit source sets
the length), VCVTPD2QQ/UQQ, VCVTPS2QQ, VCVTUDQ2PD/PS, the half-precision
VCVTPH2PS and VCVTPS2PH (the extract layout with an immediate).
## [0.13.0] — 2026-07-18
The wider AVX-512 set: ternary logic, permutes, compares, expand/compress,
the opmask instructions and the EVEX rounding/SAE/broadcast suffixes — every
encoding verified byte for byte against the Go assembler.
### Added
- `asm`: the wider EVEX/AVX-512 set, across roughly sixty new ground-truth
cases: ternary logic (VPTERNLOGD/Q), the lane shuffles/inserts/extracts
(VSHUF{F,I}{32,64}X{2,4}, the VINSERT*/VEXTRACT* {F,I}{32,64}X{2,4,8}
family, VPALIGNR), compares with an opmask destination (VCMPPD/PS/SD/SS —
a new NDS-plus-immediate form with the K register in the reg field), the
permutes (VPERMB/W, VPERMI2/T2 D/Q/PD), the wider integer families
(VPMADDWD/UBSW, VPMULHUW, VPACKSSWB/USWB/SSDW/USDW, VPABS B/W/D/Q, the
VPROL*/VPROR* rotates, the word shifts and the EVEX W1 qword shifts),
expand/compress (VEXPANDPD/PS, VPEXPANDD/Q, VCOMPRESSPD/PS, VPCOMPRESSD/
Q), the broadcasts (VPBROADCASTB/W from a GPR or memory, VBROADCASTSS/
SD), the opmask-register instructions (KAND/KOR/KXNOR/KADD/KUNPCK/KNOT/
KSHIFTL/KORTEST B/W/D/Q and KMOVQ, whose width the L/W/pp bits select),
the packed single arithmetic (VADD/VSUB/VMUL/VDIV/VMIN/VMAX PS), the
aligned moves (VMOVAPS/APD, VMOVDQA32/64, VMOVSS), the replicating moves
(VMOVSLDUP/VMOVSHDUP), the conversions (VCVTPS2DQ, VCVTTPS2DQ) and the
remaining extending and narrowing moves (VPMOVSXBW, VPMOVZXBW, VPMOVWB,
VPMOVQB).
- `asm`: the EVEX mnemonic suffixes the Go assembler accepts — the rounding
modes `.RN_SAE`, `.RD_SAE`, `.RU_SAE`, `.RZ_SAE` (the EVEX b bit with the
rounding control in L'L), suppress-all-exceptions `.SAE`, and memory
broadcast `.BCST` (the b bit, the vector length preserved, disp8×N scaled
by the element size) — each combinable with the `.Z` zeroing suffix,
validated against the Go assembler's bytes, and rejected on instructions
that do not support them.
## [0.12.0] — 2026-07-17
GOOBJ emission: gasm-assembled functions drop into a `go build` without the
Go assembler.
### Added
- `asm`: **GOOBJ object output.** `gasm asm --format goobj -p <pkgpath>`
writes the Go toolchain's own object format — the one `cmd/link` consumes
directly: the functions as non-package symbols qualified with the package
path (exactly as `cmd/asm` records assembly symbols), the `GLOBL` data,
one serialized `FuncInfo` per function (argument/frame sizes, the asm
func flag, the start line, the file table) and the four pc-value tables
(`pcsp`, `pcfile`, `pcline`, `pcinline`). The `pcsp` table carries the
real stack deltas: the assembler now tracks every stack-adjustment
boundary through the prologue (`PUSHQ BP`, `SUBQ $frame, SP`) and each
`RET`'s epilogue, so frame-pointer functions unwind correctly. The
object preamble — the version-and-experiment header the linker compares
verbatim — is captured from the installed `go tool asm`, so the output is
always consistent with the toolchain that links it.
- `asm`: relocations against file-local `GLOBL` symbols become `R_PCREL`
entries in the GOOBJ output, with the instruction's displacement field
left zero for the linker to fill (as `cmd/asm` leaves it).
### Fixed
- `parser`: 64-bit `DATA` literals above `MaxInt64`
(`DATA mask<>+8(SB)/8, $0x800f…`) parse as unsigned and keep their bit
pattern, instead of being rejected as non-integer.
### Verified
- End-to-end: a gasm-emitted GOOBJ swapped into a `go build` in place of
the toolchain's assembly object links and runs with output identical to
the baseline binary (stack-argument calls and a `GLOBL` relocation
resolved by the Go linker). All 17 go-flac AVX2 kernel functions emit as
a GOOBJ that `go tool nm` reads back with every symbol intact.
## [0.11.0] — 2026-07-16
Linkable object output: external symbols and relocatable ELF / Mach-O
objects.
### Added
- `asm`: **object-file emission.** `gasm asm --format elf` writes an
ELF64 relocatable object and `--format macho` a Mach-O x86-64
`MH_OBJECT`: a code section (`.text` / `__TEXT,__text`) and a data
section (`.data` / `__DATA,__data`), a symbol table with one symbol per
`TEXT` and `GLOBL` (file-local `<>` symbols local, the rest global), and
one PC-relative relocation per static-symbol reference
(`R_X86_64_PC32` / `X86_64_RELOC_SIGNED`, the −4 addend the form needs).
The ELF output is verified end-to-end: a gasm-emitted object links with
a C driver and runs, resolving both a file-local constant and an
external symbol; the Mach-O output is verified structurally with
`debug/macho`.
- `asm`: **external symbol references.** A reference to a symbol no
`GLOBL` in the file defines no longer aborts assembly — it is recorded
as an external relocation (`Image.Externals`, `FuncLayout.Relocs`) and
becomes an undefined global symbol in the object output. The raw image
format (`--format raw`, the default) still reports them: only an object
file can represent a reference the linker must resolve.
### Changed
- `gasm asm` takes a `--format raw|elf|macho` flag selecting what `-o`
writes; without `--format` the behaviour is unchanged (the concatenated
image).
## [0.10.0] — 2026-07-15
The EVEX floating-point and conversion set: the packed-double arithmetic,
the scalar SD/SS forms, VMOVDDUP and the width-changing conversions, each
verified byte for byte against the Go assembler.
### Added
- `asm`: the rest of the common EVEX/VEX floating-point set — packed double
arithmetic (VSUBPD, VDIVPD, VMINPD, VMAXPD, VUNPCKLPD and the EVEX form of
VUNPCKHPD), the scalar double and single operations (VSUBSD, VDIVSD,
VMINSD, VMAXSD and the full VADDSS/VSUBSS/VMULSS/VDIVSS/VMINSS/VMAXSS
family in both VEX and EVEX — the EVEX scalar forms exist for masked and
zeroing use), and VMOVDDUP (lane duplication, VEX and EVEX).
- `asm`: the width-changing conversions — VCVTDQ2PS and VCVTPS2PD (VEX and
EVEX; the destination sets the length for PS→PD), the EVEX form of
VCVTDQ2PD, and the packed-double → dword family: VCVTPD2DQ/VCVTTPD2DQ
(EVEX-512 only, a ZMM source and an XMM destination) and their X/Y
spellings (VCVTPD2DQX/Y, VCVTTPD2DQX/Y), whose length follows the wider
source — a new operand form, since the destination is always XMM while
VEX.L / EVEX.L'L ride with the source (fixed by the spelling even for a
memory source).
- `asm`: masking and zeroing on every new form — the scalar SD/SS
arithmetic, the unpacks, VMOVDDUP and the conversions all accept the
explicit K1–K7 operand and the `.Z` suffix the way Go writes them.
### Documented
- VCVTPS2PD follows the Go assembler's encoding, which omits the F3
mandatory prefix (VEX.pp / EVEX.pp = 00) that Intel's maps prescribe; the
Go toolchain's machine code is the project's byte-for-byte oracle, and
gasm reproduces it exactly (and round-trips through the x86 decoder, which
shares the convention).
### Verified
- 58 new ground-truth cases — every instruction extracted from the Go
toolchain's own assembly (go build + an executable-segment dump), checked
byte for byte and round-tripped through the decoder, covering disp8×N for
the scalar (×8/×4), duplication (×8/×32/×64) and conversion (×8/×16/×32)
memory operands, the 5-bit register fields and the masked/zeroing P2
byte. All four go-flac/go-lz4 kernels still assemble byte-identically
and lint clean.
## [0.9.0] — 2026-07-14
AVX-512 masking and a wider EVEX integer set.
### Added
- `asm`: **EVEX masking** the way Go writes it — an explicit `K1`–`K7`
operand placed among the operands (merging mask), and a `.Z` mnemonic
suffix for zeroing (`VPADDD.Z Z1, Z2, K2, Z3`). Supported across the NDS,
reg/rm, immediate-shift, align, extract, convert and move forms, including
masked comparisons with a K destination (`VPCMPEQD Z0, Z3, K2, K1`). K0 is
rejected as an explicit mask, and `.Z` without a mask is an error, matching
the Go assembler.
- `asm`: the common AVX-512 F/BW integer set — VPADDB/W, VPSUBB/W, VPANDD/Q,
VPANDND/Q, VPMULLW, VPAVGB/W, the signed/unsigned min/max family for
B/W/D/Q elements, the variable shifts VPSLLVD/Q, VPSRLVD/Q, VPSRAVD/Q, the
EVEX forms of VPSHUFD/VPSHUFB, and the VMOVDQU8/VMOVDQU16 move aliases.
Register indices 16–31 encode correctly (the mod=11 quirk carries rm[4]
in X̄). All verified byte for byte against the Go assembler.
- `lint`: masked EVEX forms (`.Z` suffix, K operands) are recognised by
`unknown-instruction` and exempted from `operand-count`.
### Fixed
- `asm`: EVEX register–register operands with indices 16–31 encoded rm[4]
into B̄ instead of X̄ (the EVEX mod=11 extension quirk), producing wrong
prefix bytes for X16+/Y16+ r/m operands.
## [0.8.0] — 2026-07-13
Standard CLI ergonomics.
### Added
- `gasm --help` prints a proper top-level help (description, commands,
flags, examples), and every subcommand now answers `-h`/`--help` with its
own usage block (usage line, description, flag defaults), exiting 0. An
unknown command points at `gasm --help` instead of dumping the whole usage.
### Changed
- The version is primarily available as the standard `gasm --version` / `-V`
flag; the `gasm version` spelling remains as an alias.
## [0.7.0] — 2026-07-12
The formatter behaves like `go fmt` and canonicalises block separation.
### Added
- `gasm fmt` now works like `go fmt`: with no arguments — or with a directory
argument — it reformats every `.s` file below it in place and lists the
changed files, skipping `.` and `_` directories (`.git`, `_refs`, …).
Explicit file arguments keep the `-w` / standard-output behaviour.
### Changed
- `format`: canonical blank-line layout — a new block (a label, `TEXT` or
`GLOBL`) is preceded by exactly one blank line, neither more nor less.
Comments leading a block stay with it (the blank line goes before them),
stacked labels share their block, the function's first label keeps hugging
its `TEXT`, and runs of blank lines collapse to one. The output remains
idempotent and round-trips through the parser. All four go-flac/go-lz4
kernels were reformatted with this release and remain byte-identical when
assembled.
## [0.6.0] — 2026-07-11
Calibrated to the Go ABI: `register-clobber` stops reporting legal code, and
the encoder learns the legacy SSE moves.
### Changed
- `lint`: **`register-clobber` is now calibrated to the Go ABI**
(`cmd/compile/abi-internal.md`), not the platform ABI. Go's stack-based
ABI0 has no System V style callee-saved registers — amd64 `BX`, `R12`–`R15`
and the arm64/riscv64/loong64 scratch sets are caller-saved or permanent
scratch, and hand-written kernels may clobber them freely. The rule now
audits only the registers Go fixes across calls: the frame pointer and the
goroutine pointer (amd64 `BP`/`R14`, arm64 `R18`/`R28`/`R29`, riscv64
`X27`, loong64 `R22`), and the goroutine pointer is reported only when the
function can reach the runtime (is not `NOSPLIT` or makes a call) — the
ABI0 transition restores it on those paths, and NOSPLIT call-free leaves
may use it, exactly as the runtime's own assembly does. Both go-flac
kernels now lint with zero diagnostics.
### Fixed
- `lint`: the liveness analysis took the destination operand to be the
*first* operand on arm64, riscv64 and loong64; Plan 9 spelling puts it last
on every architecture Go supports. The def/use and save/restore
classification on those architectures was inverted.
- `format`: a comment that follows a `RET` (typically the next function's doc
comment) is no longer indented as if it were still inside the finished
function body.
### Added
- `asm`: the legacy (non-VEX) SSE moves — `MOVOU`/`MOVO` (the Plan 9 names
for MOVDQU/MOVDQA), `MOVUPS`/`MOVAPS`/`MOVUPD`/`MOVAPD` and the scalar
`MOVSD`/`MOVSS` — and `VMOVDQU64` in the EVEX set. All verified byte for
byte against the Go assembler.
## [0.5.0] — 2026-07-10
EVEX / AVX-512: the go-flac AVX-512 kernel now assembles, byte-identically to
the Go toolchain, completing the production-kernel coverage.
### Added
- `asm`: **EVEX (AVX-512) encoding** — the four-byte EVEX prefix with the
5-bit register fields (Z0–Z31, X/Y 16–31, with the reg-r/m X̄ quirk and
V'̄ shared between vvvv and the SIB index), opmask registers (K0–K7) as
operands and as mask destinations, and the compressed disp8×N displacement
(the multiplier follows the memory operand's size, as the Go assembler's
opcode tables prescribe). Covers every AVX-512 instruction the go-flac
kernels use: VPXORD/Q, VPADDD, VPSUBD/Q, VPUNPCK*DQ, VPMULLD/Q, VPERMD,
VPSLLD/VPSRAD/VPSRAQ, VALIGND, VPCMPEQD (K destination), VMOVDQU32,
VMOVUPD, VCVTQQ2PD, VPMOVSXDQ, the narrowing stores VPMOVDW/VPMOVQD, the
extracts VEXTRACTI64X4/VEXTRACTF64X4, VFMADD231PD, VADDPD, VMULPD, the
broadcasts VPBROADCASTD/Q (GPR and memory sources take different opcodes)
and the mask moves KMOVW/KTESTW. Masking/zeroing suffixes are out of scope
— the kernels use neither.
- `asm`: `AssembleFile` now accepts file-defined global (`non-<>`) symbols
too; a reference is external only when no `GLOBL` in the file defines it.
### Fixed
- `asm`: registers X16–Y31 force the EVEX encoding of dual-form mnemonics;
previously a `VPBROADCASTD AX, Y30` fell into the VEX encoder, which cannot
represent indices above 15 and silently truncated them.
- `asm`: the VEX encoder now rejects vector register indices 16–31 instead of
encoding a truncated (wrong) register.
### Verified
- All 10 functions of the go-flac `avx512_amd64.s` kernel assemble
byte-identically to the Go toolchain's machine code (the disp32 of the one
`VMOVDQU32 idx16(SB), Z13` load is linker-filled in Go and resolved within
gasm's own image — checked to reach the right constant bytes). The AVX2
kernel's 17 functions remain byte-identical.
## [0.4.0] — 2026-07-09
The standalone assembler reaches the whole go-flac AVX2 kernel: static
symbols assemble, and all 17 kernel functions now match the Go toolchain's
machine code byte for byte.
### Added
- `asm`: **file-level assembly** — `AssembleFile` turns a parsed file into an
`Image`: the function bodies in source order followed by a data section
built from the file's `GLOBL`/`DATA` directives (each symbol 16-aligned).
- `asm`: **static-symbol (`SB`) operands** — `mask<>(SB)` references encode as
RIP-relative loads with a patched disp32, resolved against the image layout
so the output is self-consistent and position-independent. External
(non-file-local) symbols are rejected with a clear error: they need
object-file emission.
- `gasm asm` prints the data section and symbol map alongside the functions
and writes the whole image (code + data) with `-o`.
### Verified
- All 17 functions of the go-flac `avx2_amd64.s` kernel assemble
byte-identically to the Go toolchain's machine code; the only differing
bytes are the displacements of the two `VMOVDQU mask24<>(SB), X15` loads,
which the Go linker fills at link time and gasm resolves within its own
image (checked to reach the right constant bytes).
## [0.3.0] — 2026-07-08
The assembler reaches byte-identical parity with the Go toolchain on the
production go-flac AVX2 kernels: every one of the 15 kernel functions that
avoid global symbols now assembles to exactly the Go assembler's bytes (the
two holdouts load a file-local constant through `SB` and wait on relocation
support).
### Added
- `asm`: the scalar instruction families the kernels use — `CMOVcc` and
`SETcc` (conditions spelled exactly like the jumps), `LZCNT`/`TZCNT`
(legacy `F3 0F BD/BC`), the sign/zero-extending moves (`MOVBLZX`, `MOVBQZX`,
`MOVWLZX`, `MOVWQZX`, `MOVWLSX`, `MOVLQSX`), `CVTSL2SD`/`CVTSQ2SD` (the
legacy SSE encoding, as the Go assembler emits it), the traditional
three-operand `IMUL3{W,L,Q}`, and the variable-count vector shifts
(`VPSRLQ X0, Y8, Y8` — the count in an XMM register or memory takes the
ordinary NDS form).
- `asm`: **jump relaxation** — jumps start in the short (rel8) form and
expand to rel32 when the settled displacement does not fit, iterating the
layout to a fixed point (CALL is always rel32).
- `asm`: **jump-to-jump folding** — a conditional jump to a label whose only
instruction is an unconditional jump is redirected to the ultimate
target, replicating the Go toolchain's linker, which chases such chains
before it encodes branches.
- `parser`: leading negative displacements with a base and index
(`LEAQ -4(DX)(R9*4), R9`) parse into a fully populated address.
### Fixed
- `asm`: `CMP` with a register or memory operand computed **second − first**
instead of first − second, silently inverting every condition that followed
(`CMPQ SI, R10; JGE` tested R10 ≥ SI). The encoding now always records
first − second — `CMP r/m, r` with the first operand in r/m, `CMP r, r/m`
with the first operand in reg — and is byte-identical to the Go assembler.
- `asm`: register-to-register `MOV` now uses the `r/m ← r` opcode (reg =
source), the Go assembler's choice; the output is byte-identical.
## [0.2.0] — 2026-07-07
The Phase 2 assembler grows the SIMD set: shuffles, extract/insert, permute
and the moves, on top of the Phase 1 VEX forms.
### Added
- `asm`: four new VEX (AVX/AVX2) operand forms, each validated by round-trip
decoding through `golang.org/x/arch` **and** byte-for-byte against the
machine code the real Go assembler emits:
- the immediate shuffle (`VPSHUFD`, `VPERMQ`),
- the three-operand-plus-immediate form (`VSHUFPD`, `VPERM2I128`,
`VINSERTI128`),
- the lane extract (`VEXTRACTI128`, `VEXTRACTF128` — the YMM source occupies
the ModRM.reg field, the XMM/memory destination the r/m field),
- the direction-sensitive moves (`VMOVDQU`, `VMOVUPD`, `VMOVD`, `VMOVQ`,
`VMOVSD` — each direction picks its own opcode and VEX.W; a vector→vector
move uses the store-form layout, matching the Go assembler),
- the no-operand `VZEROUPPER`, and `VPERMD` in the NDS form,
- the floating-point and FMA set (`VADDPD`, `VMULPD`, `VXORPD`,
`VUNPCKHPD`, the scalar `VADDSD`/`VMULSD`, `VCVTDQ2PD`, `VFMADD231PD`).
With the scalar set and the earlier NDS / reg-rm / immediate-shift forms,
the encoder now covers every integer, shuffle and FP instruction the
go-flac AVX2 kernels use.
- `asm`: `CMP` accepts the immediate in the second operand position
(`CMPL CX, $31`) — the spelling the Go assembler accepts — encoding it
identically to the immediate-first form.
### Fixed
- `asm`: an unused VEX.vvvv field is now stored as `1111` (v̄vvv = 1111), as
the hardware requires — the previous value (`0000`) made the two-operand
reg/rm forms (VPMOVSXWD, VPBROADCASTD, VMOVMSKPS, …) raise #UD on real CPUs
and differ from the Go assembler's bytes. The round-trip decoder ignores
the field on these instructions, which is why the byte-for-byte Go
comparison (added this release) is now part of the test suite.
## [0.1.0] — 2026-07-06
Initial release — the Phase 1 foundation.
### Added
- `token`, `lexer`, `ast`, `parser`: a hand-written, error-tolerant front end
for Plan 9 assembly. The lexer splices C-preprocessor line continuations
(`\` before a newline) so multi-line `#define` macros parse as one opaque
directive. Validated against the production AVX2/AVX-512 kernels in
`go-libraries/go-flac` and the Go runtime's `src/runtime/*.s` for all four
architectures, with zero parse errors.
- `arch`: register files and **complete** instruction tables for amd64,
arm64, riscv64 and loong64, with the middle-dot symbol separator and static
(`<>`) symbols. Instruction names are generated from the Go toolchain's own
assembler source (`just gen`) — the `anames` opcode lists plus the common
opcodes and the per-architecture front-end aliases (arm64 `B`/`BL`, the
`.P`/`.W` addressing suffixes, loong64 `JAL`, the x86 conditional-jump
spellings) — so every mnemonic the real assembler accepts is recognised.
- `lint`: conservative rules — `unknown-instruction`, `operand-count`,
`undefined-label`, `duplicate-label`, `missing-ret`,
`missing-textflag-include`, `abi-argsize` and `unreachable-code`. Macro
invocations are recognised (in-file `#define` names and underscore
identifiers) and the label/RET heuristics are suppressed in macro-using
files. `abi-argsize` parses the `// func` signature with the Go parser and
checks the declared TEXT argument size against Go's ABI0 layout;
`unreachable-code` flags dead code after `RET`, suppressed where reachability
is undecidable (PC-relative jumps, register-indirect branches, `#ifdef`).
Register liveness is computed by dataflow over the control-flow graph (basic
blocks, def/use, iterative backward iteration) and drives `register-clobber`,
an audit that flags a callee-saved register written but never saved/restored.
`funcdata-pcdata` validates the structure of `FUNCDATA`/`PCDATA` directives.
Zero error-severity diagnostics across the 90-file Go runtime corpus and the
production go-flac kernels (the `register-clobber` audit additionally reports
the go-flac kernels' unsaved callee-saved register use for review).
- `format`: an idempotent canonical formatter (operand spacing and per-function
mnemonic alignment) that preserves comments and round-trips through the
parser.
- `lsp`: a Language Server Protocol server over stdio providing completion,
hover documentation, document symbols, publish-diagnostics and semantic-token
highlighting.
- `asm`: a standalone amd64 (x86-64) assembler — an instruction encoder (REX/
ModR-M/SIB/displacement/immediate plus the scalar instruction set, and VEX/
AVX2 SIMD across three operand forms — NDS, reg/rm and immediate-shift —
covering the bulk of the integer SIMD set) validated by round-trip decoding
against `golang.org/x/arch`, and an assembler that drives the parser's AST
into the encoder with local-label resolution and `FP`/`SP` frame mapping
(plus Go prologue/epilogue generation), producing output byte-identical to the
Go assembler for the supported operand forms.
- `cmd/gasm`: the `gasm` binary with `tokens`, `parse`, `fmt`, `lint`, `asm`
and `lsp` subcommands.
- `_gen`: the generator that rebuilds the architecture instruction tables from
the Go toolchain source (`just gen`).
+94
View File
@@ -0,0 +1,94 @@
# Contributing to gasm-devkit
## Prerequisites
- Go 1.26 or later (`toolchain go1.26.5`)
- `just` command runner
- A Linux, FreeBSD, or macOS host on amd64 or arm64
## Development Setup
```sh
git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git
cd gasm-devkit
just install # download module dependencies
just build # go vet + gofmt check
just test # full test suite with race detector
```
## Commands
Every just recipe:
| Recipe | What it does |
|--------|-------------|
| `just` | List all recipes |
| `just install` | `go mod download` |
| `just build` | `go vet ./...` + `gofmt -l .` check — zero errors required |
| `just test` | `go test -race -count=1 -coverprofile=coverage.out ./...` + 80 % coverage gate |
| `just fmt` | `gofmt -w .` |
| `just run -- lint file.s` | Run the CLI with `go run` (args after `--`) |
| `just install-bin` | Install `gasm` into `$GOBIN` with the release version stamped |
| `just gen` | Regenerate `arch/*_gen.go` instruction tables from the Go toolchain |
| `just uninstall` | Remove build artefacts (`coverage.out`, `gasm`, `*.test`) |
## Running a Single Test
```sh
go test -run TestVexGroundTruth ./asm/
go test -run TestDifferentialLZ4Fuzz ./verify/
```
## Testing the Debugger
The interactive debugger (`gasm debug`) requires a compiled binary —
`go run` does not work for the child process. Install first:
```sh
just install-bin
gasm debug --func add testdata/verify/basic_amd64.s
```
## Code Style
See [AGENTS.md](AGENTS.md) for the full style guide. Key points:
- `gofmt` — zero diff.
- `go vet` — zero warnings.
- Standard library only in production code; `golang.org/x/arch` in tests.
- No cgo, no C, no JavaScript.
- Hand-written Plan 9 assembly; tables generated only via `_gen/gen.go`.
## Branches and Releases
- `development` is the working branch.
- `main` is release-only: `git merge --ff-only development`, then `git tag vX.Y.Z`.
- Conventional Commits: `feat(asm): add EVEX gather and scatter`.
- Every commit ends with `Assisted-by: <model-name>`.
## CI
There is no CI pipeline in this repository. The Definition of Done
(`just build` + `just test` + `just fmt`) is enforced locally.
## AI-Assisted Contributions
AI agents may assist with code, documentation, tests, and review. All
AI-assisted changes must:
- Include the trailer `Assisted-by: <model-name>` in the commit message
(e.g. `Assisted-by: DeepSeek V4 Pro`).
- Follow the [AGENTS.md](AGENTS.md) rules.
- Pass the Definition of Done before committing.
Attribute agent authorship in issues and pull requests on one trailing
line:
```
_Assisted-by: DeepSeek V4 Pro_
```
## Questions
Open an issue at
[sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit/issues).
+352
View File
@@ -0,0 +1,352 @@
# gasm-devkit
Developer tooling for **GAsm** — Go's built-in Plan 9 assembler.
[sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
Go ships an assembler but no tooling for it. There is no syntax highlighting,
no autocomplete, no linter, no static analyser, no formatter, no standalone
assembler and no debugger for `.s` files. Developers write assembly blind,
validate it by benchmark, and debug it by print statement.
gasm-devkit is the missing toolkit. It is a single, self-contained binary —
`gasm` — that brings proper developer tooling to Plan 9 assembly:
```
gasm tokens dump the lexical token stream
gasm parse parse and report syntax errors
gasm fmt canonicalise formatting (gofmt for assembly)
gasm lint static checks
gasm lsp language server (completion, hover, symbols, diagnostics, highlighting)
gasm asm standalone assembler (Phase 2)
gasm verify dynamic analysis & verification (Phase 3)
gasm debug source-level debugger (Phase 4)
```
> **Status: Phase 3 — done.** Phase 1 (the language foundation,
> linter, formatter and language server) shipped in v0.1.0; Phase 2 (the
> standalone assembler — the full amd64 instruction set plus ELF, Mach-O
> and GOOBJ object emission) in v0.12.0; Phase 3 (dynamic analysis —
> JIT execution, differential testing, ABI checks and coverage profiling)
> in v0.25.0. See [Roadmap](#roadmap).
## Architecture support
gasm-devkit targets every architecture Go's assembler speaks. The instruction
tables are **generated from the Go toolchain's own assembler source**
(`cmd/internal/obj/<arch>`), so gasm-devkit recognises *every* mnemonic the
real assembler accepts — not a hand-maintained subset that drifts and rots.
| Architecture | GOARCH | File suffix | Instructions recognised |
|--------------|-------------|----------------|------------------------------------|
| AMD64 | `amd64` | `_amd64.s` | 1600 + common opcodes + traditional aliases |
| ARM64 | `arm64` | `_arm64.s` | 538 + common opcodes |
| RISC-V | `riscv64` | `_riscv64.s` | 961 + common opcodes |
| LoongArch | `loong64` | `_loong64.s` | 799 + common opcodes |
"Common opcodes" are the instructions shared by every architecture (`RET`,
`JMP`, `NOP`, `CALL`, `TEXT`, `FUNCDATA`, `PCDATA`, …). AMD64 additionally
carries the traditional conditional-jump spellings (`JZ`, `JNZ`, `JA`, `JC`,
…) that the assembler accepts as aliases. Regenerating the tables is one
command — `just gen` — and requires only a Go installation; the committed
output has no runtime dependency on the toolchain.
The target *architectures* above are what the toolkit analyses. The toolkit
itself is portable Go and builds on Linux, FreeBSD and macOS, on amd64 and
arm64 hosts.
## Roadmap
The work is delivered in four phases. Each phase is completed and hardened
before the next begins. The ordering follows a dependency chain: understand
the code statically (Phase 1), make it runnable (Phase 2), then run it and
observe or control it (Phases 3–4).
### Phase 1 — language foundation, editor tooling and static analysis · *done*
Everything needed to read, understand, check, format and highlight GAsm —
without executing it.
| Capability | Status |
|------------|--------|
| Lexer — permissive, position-aware scanner for all four architectures | done |
| Parser — line-oriented, error-tolerant, full AST with source positions | done |
| Instruction + register tables for amd64, arm64, riscv64, loong64 (generated, complete) | done |
| Linter — `unknown-instruction`, `operand-count`, `undefined-label`, `duplicate-label`, `missing-ret`, `missing-textflag-include`, `abi-argsize`, `unreachable-code`, `register-clobber`, `funcdata-pcdata` | done |
| Formatter — idempotent, comment-preserving, per-function alignment; a `RET` terminates the body for indentation, so the next function's doc comment stays at column 0; exactly one blank line before every block (label, `TEXT`, `GLOBL`) and runs of blanks collapsed; directory / no-argument mode reformats every `.s` in place, `go fmt`-style | done |
| Language server — completion, hover, document symbols, diagnostics, semantic-token highlighting | done |
| CLI — `gasm tokens / parse / fmt / lint / lsp` | done |
| Real-world validation against production AVX2 / AVX-512 kernels | done |
| Lint hardening — zero false positives across the Go runtime corpus (90 files, all four architectures): macro-invocation handling, branch aliases (`B`/`BL`/`JAL`), addressing suffixes (`.P`/`.W`), terminal `UNDEF` | done |
| Static analysis — `abi-argsize` (argument/result area computed from the `// func` signature under Go's ABI0 layout and checked against the TEXT declaration) and `unreachable-code` (dead code after `RET`, suppressed where reachability is undecidable: PC-relative jumps, register-indirect branches, `#ifdef`) | done |
| Static analysis — register liveness (CFG construction + per-instruction def/use + iterative backward dataflow) driving `register-clobber`, calibrated to the **Go ABI** (not System V): flags writes to the registers Go fixes across calls — the frame pointer and the goroutine pointer (`R14` on amd64, `R28`/`R29` on arm64, `X27` on riscv64, `R22` on loong64, plus the OS-reserved `R18` on arm64) — that are never saved/restored; the goroutine pointer is reported only when the function can reach the runtime (not `NOSPLIT`, or makes calls), matching how the runtime's own assembly uses it. `funcdata-pcdata` structural validation of `FUNCDATA`/`PCDATA` operands and indices | done |
> **Limitation — macros.** gasm-devkit reads `.s` source as written; it does
> **not** run the C preprocessor, so `#define` macros are not expanded. Files
> that use macros (the runtime's `asm_*.s`, `race_*.s`, `sys_*.s`, …) parse
> cleanly, and macro *invocations* are recognised and never flagged, but the
> `undefined-label` and `missing-ret` heuristics are suppressed in macro-using
> files because labels a macro defines are invisible without expansion. Full
> macro expansion is future work (it pairs naturally with the Phase 2
> assembler). Hand-written, macro-free kernels — such as everything in
> `go-libraries` — are analysed in full.
### Phase 2 — standalone assembler · *done*
Assembly without the Go toolchain in the loop.
- **`gasm asm`:** a standalone assembler that turns a `.s` file into machine
code directly — pure Go, no `go build`, no external toolchain. Useful for
fast iteration, for environments without a full Go installation, and as the
execution substrate that Phases 3 and 4 build on.
Done so far:
- An amd64 (x86-64) **instruction encoder** — REX/ModR-M/SIB/displacement/
immediate machinery and the scalar instruction set (MOV, the ALU group, TEST,
LEA, INC/DEC/NEG/NOT, shifts, IMUL and IMUL3, PUSH/POP, JMP/CALL/Jcc,
CMOVcc, SETcc, LZCNT/TZCNT, the sign/zero-extending moves — MOVBLZX and
friends, MOVLQSX — and CVTSL2SD/CVTSQ2SD), validated by round-tripping
every encoding through `golang.org/x/arch`'s decoder and byte-for-byte
against the Go assembler.
- An **assembler** that drives the parser's AST into the encoder with local-
label resolution — jumps start in the short (rel8) form and expand to rel32
when the displacement does not fit, and jump-to-jump chains are folded the
way the Go toolchain folds them — so `gasm asm <file>` emits machine code
for each `TEXT` function.
- **File-level assembly with static data** — `GLOBL`/`DATA` symbols are laid
out in a data section behind the code and references to them (`mask<>(SB)`)
are encoded RIP-relative with the displacement resolved within the image,
so the output is self-consistent and position-independent. References to
symbols no `GLOBL` in the file defines are recorded as relocations and
carried into the object-file output.
- **GOOBJ emission** — `gasm asm --format goobj -p <pkgpath>` writes the Go
toolchain's own object format (the one `cmd/link` consumes directly), so
gasm-assembled kernels drop into a `go build` without the Go assembler:
the functions as non-package symbols, `GLOBL` data, one `FuncInfo` per
function and the pc-value tables (`pcsp` with the real prologue/epilogue
stack deltas, `pcfile`, `pcline`, `pcinline`). Verified end-to-end by
swapping a gasm-emitted object into a `go build` in place of the
toolchain's, linking and running — bit-identical behaviour.
- **Object-file emission** — `gasm asm --format elf` / `--format macho`
writes a relocatable object (a `.text` and a `.data` section, a symbol
table — file-local `<>` symbols local, the rest global — and one
`R_X86_64_PC32` / `X86_64_RELOC_SIGNED` relocation per static-symbol
reference) that links with the system toolchain: external references
resolve against undefined symbols, file-local ones against the data
section. Verified end-to-end by linking a gasm-emitted object with a C
driver and running it.
- **`FP`/`SP` frame mapping** — the pseudo-registers are translated onto the
hardware stack pointer (`x+N(FP)` → `(N+8)(SP)` for a zero frame, `(N+frame+
16)(SP)` with a frame pointer; locals via `x-N(SP)`), and the Go-style
prologue/epilogue is generated for functions with a frame. The output is
**byte-identical to the Go assembler** for these cases (verified against
`go tool objdump`).
- **SIMD (VEX / AVX2)** — the VEX prefix machinery (2-byte C5 and 3-byte C4)
with XMM/YMM vector registers, validated by round-trip decoding **and**
byte-for-byte against the Go assembler's machine code, across eight operand
forms: the three-operand NDS form (VPADDD/Q, VPSUBD/Q, VPXOR, VPOR, VPAND/N,
VPCMPEQD, VPCMPGTQ, VPUNPCK*, VPMULLD, VPMULDQ, VPSHUFB, VPACKSSDW,
VPERMD), the two-operand reg/rm form (VPMOVSXWD/DQ, VPMOVZXDQ,
VPBROADCASTD/Q, VPMOVMSKB, VMOVMSKPS, VCVTDQ2PD), the immediate-shift and
variable-count shifts (VPSLLD/Q, VPSRAD, VPSRLD/Q with an immediate or an
XMM/memory count), the immediate shuffle (VPSHUFD, VPERMQ), the
three-operand-plus-immediate form (VSHUFPD, VPERM2I128, VINSERTI128), the
lane extract (VEXTRACTI128, VEXTRACTF128), the direction-sensitive moves
(VMOVDQU, VMOVUPD, VMOVD, VMOVQ, VMOVSD), the no-operand VZEROUPPER, and
the floating-point set: the packed double arithmetic
(VADDPD/VSUBPD/VMULPD/VDIVPD/VMINPD/VMAXPD), the unpacks
(VUNPCKHPD/VUNPCKLPD), the scalar SD and SS operations, VMOVDDUP, the
width-changing conversions (VCVTDQ2PS, VCVTPS2PD, VCVTDQ2PD and the
VCVTPD2DQX/Y / VCVTTPD2DQX/Y spellings, whose VEX.L follows the wider
source) and VFMADD231PD.
- **SIMD (EVEX / AVX-512)** — the four-byte EVEX prefix with the 5-bit
register fields (Z0–Z31, X/Y 16–31), opmask registers (K0–K7 as operands
and mask destinations, KMOVW, KTESTW) and the compressed disp8×N
displacement, covering every AVX-512 instruction the go-flac kernels use:
VPXORD/Q, VPADDD, VPSUBD/Q, VPUNPCK*DQ, VPMULLD/Q, VPERMD, VPSLLD/VPSRAD/
VPSRAQ, VALIGND, VPCMPEQD (with a K destination), VMOVDQU32, VMOVUPD,
VCVTQQ2PD, VPMOVSXDQ, the narrowing stores VPMOVDW/VPMOVQD, the lane
extracts VEXTRACTI64X4/VEXTRACTF64X4, VFMADD231PD, VADDPD, VMULPD,
VMOVDQU64 and the broadcasts VPBROADCASTD/Q from a GPR or memory, plus the
wider AVX-512 F/BW integer set (VPADDB/W, VPSUBB/W, VPANDD/Q/ND/NQ, VPMULLW,
VPMIN*/VPMAX* for B/W/D/Q elements, signed and unsigned, VPAVGB/W, the variable
shifts VPSLLV*/VPSRLV*/VPSRAV*, VMOVDQU8/16), the common floating-point
and conversion set (the packed double and single arithmetic
VADD/VSUB/VMUL/VDIV/VMIN/VMAX PD and PS, the scalar SD/SS operations —
whose EVEX forms exist for masked and zeroing use — the VUNPCK{L,H}PD
unpacks, VMOVDDUP, VMOVSLDUP/VMOVSHDUP and the VCVT* conversions), and
the wider AVX-512 set: ternary logic (VPTERNLOGD/Q), lane shuffles,
inserts and extracts (VSHUF{F,I}{32,64}X{2,4}, the VINSERT*/VEXTRACT*
{F,I}{32,64}X{2,4,8} family, VPALIGNR), compares with an opmask
destination (VCMPPD/PS/SD/SS), the permutes (VPERMB/W, VPERMI2/T2
D/Q/PD), the wider integer families (VPMADDWD/UBSW, VPMULHUW, VPACK*,
VPABS*, the VPROL*/VPROR* rotates and the word shifts), expand/compress
(VEXPAND*/VCOMPRESS*, VPEXPAND*/VPCOMPRESS*), the broadcasts
(VPBROADCASTB/W, VBROADCASTSS/SD), the opmask instructions (KAND/KOR/
KXNOR/KADD/KUNPCK/KNOT/KSHIFTL/KORTEST, KMOVQ), the aligned moves
(VMOVAPS/APD, VMOVDQA32/64, VMOVSS) and the remaining extending and
narrowing moves, the floating-point helper and conversion tail
(VRCP14*, VRSQRT14*, VGETEXP*, VGETMANT*, VSCALEF*, VRNDSCALE*,
VREDUCE*, VFIXUPIMM*, VRANGE*, VFPCLASS* with a K destination, and the
VCVT* conversions VCVTQQ2PS, VCVTPD2QQ/UQQ, VCVTPS2QQ, VCVTUDQ2PD/PS,
VCVTPH2PS, VCVTPS2PH), and gather/scatter with VSIB addressing
(VGATHER*/VPGATHER* in both the VEX mask-register spelling and the EVEX
K-mask spelling — where the L'L field follows the VSIB index — plus
VSCATTER*/VPSCATTER*). The EVEX mnemonic suffixes the Go assembler
accepts are honoured: rounding modes (.RN_SAE, .RD_SAE, .RU_SAE,
.RZ_SAE), suppress-all-exceptions (.SAE) and memory broadcast (.BCST,
with the element-sized disp8×N), each combinable with the .Z zeroing
suffix. Masking is supported the way
Go writes it — an explicit K1–K7 operand placed among the operands, and a
`.Z` mnemonic suffix for zeroing.
- **Legacy SSE moves** — `MOVOU`/`MOVO` (the Plan 9 names for MOVDQU/MOVDQA),
`MOVUPS`/`MOVAPS`/`MOVUPD`/`MOVAPD` and the scalar `MOVSD`/`MOVSS`.
- **Both go-flac kernels — all 17 AVX2 and all 10 AVX-512 functions —
assemble byte-identically to the Go toolchain's machine code**; the only
differing bytes are the displacements of the static-constant loads, which
the Go linker fills at link time and gasm resolves within its own image
(verified to reach the right constant bytes).
Remaining for Phase 2:
- External (cross-package) symbol references in the GOOBJ output —
**deferred** with a recorded decision and three options; see
[`docs/DEFERRED.md`](docs/DEFERRED.md). Single-package objects (no
cross-package references) work today, which covers the production
kernels. With that item deferred, the amd64 instruction set — scalar,
VEX/AVX2 and the full EVEX/AVX-512 set including GPR-interchanging
conversions — is complete.
### Phase 3 — dynamic analysis · *done*
Run the code and check what static analysis cannot. The oracle is the
portable Go implementation every kernel is derived from.
- **`gasm verify`:**
- **JIT execution substrate** — *done.* Assemble the kernel, map it into
executable memory (`syscall.Mmap`, W^X) and call it through an ABI0
trampoline; pure Go, no cgo, no external toolchain. Both go-lz4 kernels
(AVX2, 845 bytes total) JIT-load and execute correctly.
- **Differential testing** — *done.* The JIT-assembled kernel is fuzzed
with random valid LZ4 blocks and hostile garbage, comparing the result
**bit-for-bit** against a portable Go reference; the automated form of
the project's bit-identical contract.
- **Runtime ABI checks** — *done.* The ABI-checking trampoline sets
sentinels in BP and R14, verifies they survive the call, and fills a
128-byte red-zone canary below SP; both go-lz4 kernels pass clean.
- **Coverage / basic-block profiling** — *done.* Static block enumeration
from the assembler's label map (27 blocks in `decodeBlockAVX2`) plus
multi-input path-diversity measurement: how many observationally distinct
execution paths a test corpus exercises.
### Phase 4 — debugger · *in progress*
- **`gasm debug`:** single-step a GAsm function, inspect registers, set
breakpoints on labels, and hex-dump memory — the interactive counterpart
to Phase 3's execution substrate.
- **MVP** — *done.* ptrace-based debuggee subprocess (PTRACE_TRACEME +
LockOSThread), entry breakpoint (auto-run to function start),
single-step, register inspection, label resolution, breakpoint
management via `/proc/pid/mem`, and an interactive REPL.
- **Remaining:** disassembly at PC (x86asm decode), memory-write support,
watchpoints, source-line mapping, and multi-platform support
(FreeBSD/macOS ptrace variants).
### Phase 5 — the other architectures
- **Encoding for arm64, riscv64 and loong64.** The lexer, parser, linter
and formatter already cover all four architectures; the assembler today
encodes amd64 only. Phase 5 brings the same encode-and-verify treatment
(instruction tables already generated from the toolchain, every encoding
checked byte for byte against `go tool asm`) to the remaining three.
## Principles
- **Pure Go and GAsm only.** No C, no cgo, no external toolchains, no native
binaries, no JavaScript runtimes. The parser is hand-written; there is no
parser generator.
- **Self-contained.** The toolkit's production code depends only on the
standard library; one binary, no runtime data files. The single module
dependency, `golang.org/x/arch`, is used **only in tests** to validate the
instruction encoder by round-trip decoding — it is never linked into the
`gasm` binary.
- **Portable.** Builds and runs on Linux, FreeBSD and macOS; amd64 and arm64
hosts. Latest stable Go only.
- **No vendor lock-in.** The integration surface is the Language Server
Protocol and a command-line interface — both open standards. No cloud
service, no proprietary API, no dependence on any one editor's internals.
- **Complete and verifiable.** Instruction coverage is generated from the
assembler's own source and regenerated on demand, so it cannot silently fall
behind the toolchain.
## Components
| Package | Purpose |
|---------|---------|
| `token` | Lexical token kinds and source positions. |
| `lexer` | Hand-written scanner for Plan 9 assembly. |
| `ast` | The abstract syntax tree. |
| `parser` | Line-oriented, error-tolerant parser producing the AST. |
| `arch` | amd64, arm64, riscv64 and loong64 register files and instruction tables. |
| `lint` | Conservative static checks. |
| `format` | A canonical formatter — `gofmt` for assembly. |
| `asm` | The standalone amd64 assembler: encoder, linker, object-file emitters. |
| `verify` | JIT execution substrate for dynamic analysis (Phase 3). |
| `debug` | Interactive ptrace debugger for amd64 (Phase 4). |
| `lsp` | Language Server Protocol server. |
| `cmd/gasm` | The `gasm` binary tying it all together. |
| `_gen` | The generator that rebuilds the instruction tables from the Go toolchain. |
See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the design rationale and
data flow, [`docs/ZED.md`](docs/ZED.md) for the editor-integration story, and
[`docs/DEFERRED.md`](docs/DEFERRED.md) for design decisions deliberately
postponed (with the analysis needed to pick them up again).
## Quick start
```sh
just install # download dependencies (there are none)
just build # go vet + gofmt check — zero errors, zero warnings
just test # full suite, race detector, 80 % coverage gate
just fmt # gofmt the tree
just gen # regenerate the instruction tables from the Go toolchain
```
Install the binary and use it:
```sh
just install-bin # installs gasm into $GOBIN
gasm --help # overview of commands and flags
gasm tokens kernel_amd64.s # dump the token stream
gasm parse kernel_amd64.s # parse, report syntax errors
gasm fmt -w kernel_amd64.s # canonicalise in place
gasm fmt # reformat every .s below here, like go fmt
gasm lint *.s # static checks
gasm asm --format elf -o k.o k.s # assemble to a linkable ELF object
gasm verify kernel_amd64.s # JIT-load and report functions
gasm verify --ground-truth k.s # byte-for-byte vs go tool asm
gasm debug --func name k.s # interactive debugger
```
See [CONTRIBUTING.md](CONTRIBUTING.md) for the full development workflow,
[docs/cli.md](docs/cli.md) for the command reference, and
[docs/development.md](docs/development.md) for setup and recipes.
## Editor integration
`gasm lsp` speaks the Language Server Protocol over standard input/output, so
any LSP-capable editor can use it — point your editor's LSP client at the
binary and associate it with `.s` files. Syntax highlighting is delivered as
**LSP semantic tokens**, so no editor-specific grammar is required. The server
infers the target architecture from the file-name suffix
(`_amd64.s` / `_arm64.s` / `_riscv64.s` / `_loong64.s`).
Zed users should read [`docs/ZED.md`](docs/ZED.md): Zed's native highlighting
engine (Tree-sitter, C/WASM) cannot be fed from pure Go, so the pure-Go path
into Zed is the language server and its semantic tokens.
## Licence
BSD-3-Clause — the same licence as Go itself. See [`LICENSE`](LICENSE).
+14
View File
@@ -2,6 +2,8 @@
How gasm-devkit is put together and why.
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
## Design goals
1. **A real AST, not a grammar hack.** The linter, analyser, assembler and
@@ -309,6 +311,18 @@ The `gasm verify` CLI subcommand exposes this: it loads a file, reports the
available functions and (with `-smoke`) calls each NOSPLIT function with zeroed
arguments to confirm the trampoline round-trips.
### `debug`
The interactive debugger (Phase 4, linux/amd64). It launches the target
function in a child process that maps the JIT code, calls
`PTRACE_TRACEME`, and stops; the parent attaches via ptrace and controls
execution. Breakpoints are patched as INT3 bytes through `/proc/pid/mem`
(PTRACE_PEEKTEXT is unreliable with Go's multi-threaded runtime).
The child pins its goroutine to the OS thread with `runtime.LockOSThread`
so the traced thread is the one executing JIT code. The REPL provides
single-step, register inspection, label resolution, and breakpoint
management.
## Extension points
- **New architecture:** add an entry to the generator in `_gen`, run
+110
View File
@@ -0,0 +1,110 @@
# CLI Reference
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
`gasm` is a single binary with subcommands. Run `gasm --help` for an
overview, or `gasm <command> -h` for a command's usage and flags.
## Global Flags
| Flag | Description |
|------|-------------|
| `-h`, `--help` | Show help |
| `-V`, `--version` | Print the version |
## `gasm tokens <file>`
Print the lexical token stream of FILE: position, token kind, and text,
one token per line. FILE may be `-` to read standard input.
## `gasm parse <file>`
Parse FILE and report syntax errors on stderr. On success, prints how
many declarations and TEXT functions the file contains.
## `gasm fmt [-w] [path...]`
Canonicalise the formatting of Plan 9 assembly sources: indentation,
operand spacing, per-function mnemonic alignment, and blank-line layout.
| Flag | Description |
|------|-------------|
| `-w` | Write result to the source file (default: print to stdout) |
With no arguments, or with a directory argument, every `.s` file below
it is reformatted in place and the names of changed files are listed
(`go fmt` style). `.` and `_` directories are skipped.
## `gasm lint <file...>`
Run static checks and print diagnostics as
`file:line:col: severity: message [code]`. Exit status is non-zero when
an error-severity diagnostic is found.
| Flag | Description |
|------|-------------|
| `-disable` | Comma-separated rule codes to disable |
Rules: `unknown-instruction`, `operand-count`, `undefined-label`,
`duplicate-label`, `missing-ret`, `missing-textflag-include`,
`abi-argsize`, `unreachable-code`, `register-clobber`,
`funcdata-pcdata`.
## `gasm asm [--format raw|elf|macho|goobj] [-p pkg] [-o out] <file>`
Assemble FILE (amd64) to machine code.
| Flag | Description |
|------|-------------|
| `--format` | Output format: `raw` (default), `elf`, `macho`, `goobj` |
| `-p` | Package path (required for `--format goobj`) |
| `-o` | Write output to file (default: hex dump to stdout) |
## `gasm verify [flags] <file.s>`
Assemble FILE, map it into executable memory, and run dynamic checks.
| Flag | Description |
|------|-------------|
| `--ground-truth` | Compare machine code byte-for-byte against `go tool asm` |
| `--fuzz` | Differential fuzz: JIT both gasm and go-tool-asm, compare outputs |
| `-n` | Fuzz iterations per function (default: 1000) |
| `--abi` | Run ABI-checking calls (sentinel registers + red zone) |
| `--profile` | List basic-block structure per function |
| `--smoke` | Call each NOSPLIT function with zeroed args |
The `--fuzz` mode runs each function in a subprocess; a partial function
(e.g. a decoder that faults on malformed input) is reported as
`CRASH` without killing the parent. Use `--ground-truth` for decoders.
## `gasm debug --func <name> <file.s>`
Interactive debugger for JIT-assembled amd64 functions. Requires a
compiled binary on `$PATH` (not `go run`).
| Flag | Description |
|------|-------------|
| `--func` | Function to debug (required) |
REPL commands:
| Command | Description |
|---------|-------------|
| `break <label\|addr>` | Set a breakpoint |
| `step [n]` | Single-step n instructions |
| `continue` | Run until next breakpoint or exit |
| `regs` | Print general-purpose registers |
| `x [addr] [len]` | Hex-dump memory |
| `labels` | List function labels and offsets |
| `quit` | Kill the debuggee and exit |
## `gasm lsp`
Run the language server over standard input/output (JSON-RPC 2.0 with
Content-Length framing). Point an LSP-capable editor at the binary and
associate it with `.s` files. The target architecture is inferred from
the file-name suffix (`_amd64.s`, `_arm64.s`, `_riscv64.s`,
`_loong64.s`).
Provides: completion, hover, document symbols, diagnostics, and
semantic-token highlighting.
+108
View File
@@ -0,0 +1,108 @@
# Development Guide
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
## Prerequisites
- **Go** 1.26+ with `toolchain go1.26.5`
- **just** — the command runner; every task below is a just recipe
- No external dependencies beyond the Go toolchain
## Quick Start
```sh
git clone https://sourcedock.dev/petrbalvin/gasm-devkit.git
cd gasm-devkit
just install # go mod download
just build # go vet + gofmt — must pass with zero output
just test # full suite, race detector, 80 % coverage gate
```
## Just Recipes
### `just build`
Runs `go vet ./...` and checks `gofmt -l .` produces no output. This is
the minimum bar before any commit.
### `just test`
```sh
go test -race -count=1 -coverprofile=coverage.out ./...
```
Plus an `awk` gate that fails if total coverage is below 80 %.
### `just fmt`
```sh
gofmt -w .
```
Run after editing any Go source. The output must be idempotent.
### `just run -- <args>`
Runs the CLI via `go run` with the version string stamped:
```sh
just run -- lint kernel_amd64.s
just run -- fmt -w kernel_amd64.s
just run -- verify --ground-truth kernel_amd64.s
```
### `just install-bin`
Installs the `gasm` binary into `$GOBIN` with the release version
embedded via `-ldflags "-X main.version=..."`.
### `just gen`
Regenerates the architecture instruction tables in `arch/` by parsing
the Go toolchain's own assembler source
(`$GOROOT/src/cmd/internal/obj/<arch>/anames.go`). Requires a Go
installation. Output is committed — no runtime dependency on the
toolchain.
### `just uninstall`
Removes `coverage.out`, the `gasm` binary, and `*.test` artefacts.
## Running Individual Tests
```sh
go test -run TestVexGroundTruth ./asm/
go test -run TestDifferentialLZ4Fuzz ./verify/
go test -run TestFLACDecorrelate ./verify/
go test -run TestGOObjectLinkAndRun ./asm/
```
## Debugger Note
`gasm debug` spawns a child process from the binary on `$PATH`. It does
not work with `go run` — install first:
```sh
just install-bin
gasm debug --func decodeBlockAVX2 path/to/kernel_amd64.s
```
## Project Layout
```
cmd/gasm/ CLI entry point (subcommands)
token/ Lexical token kinds and positions
lexer/ Hand-written scanner
ast/ Abstract syntax tree
parser/ Line-oriented parser
arch/ Register and instruction tables (generated)
lint/ Static analysis rules
format/ Canonical formatter
lsp/ Language Server Protocol server
asm/ Standalone assembler, encoder, object emitters
verify/ JIT execution, differential testing, ABI checks
debug/ Interactive ptrace debugger (linux/amd64)
_gen/ Instruction table generator
testdata/ Test fixtures
docs/ Architecture, development, CLI reference
```