Assisted-by: GLM 5.3 Flash
This commit is contained in:
+98
-13
@@ -4,7 +4,9 @@ How gasm-devkit is put together and why.
|
||||
|
||||
Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrbalvin/gasm-devkit)
|
||||
|
||||
## Design goals
|
||||
## Overview
|
||||
|
||||
Three design goals shape everything below.
|
||||
|
||||
1. **A real AST, not a grammar hack.** The linter, analyser, assembler and
|
||||
language server all need to *reason* about assembly, not just colour it.
|
||||
@@ -20,10 +22,10 @@ Repository: [sourcedock.dev/petrbalvin/gasm-devkit](https://sourcedock.dev/petrb
|
||||
through two vendor-neutral interfaces: a CLI and an LSP server. No editor
|
||||
owns the toolkit; the toolkit is offered to editors on standard terms.
|
||||
|
||||
## Pipeline
|
||||
The components, and how data moves between them:
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
flowchart TD
|
||||
SRC["source .s"] --> LEX["lexer<br/>token stream"]
|
||||
LEX --> PAR["parser<br/>AST + diagnostics"]
|
||||
LEX --> FMT["format<br/>re-space tokens"]
|
||||
@@ -42,9 +44,34 @@ graph TD
|
||||
|
||||
The lexer is the shared foundation: the parser builds the AST from it, the
|
||||
formatter re-spaces its tokens directly, and the language server uses it for
|
||||
semantic highlighting.
|
||||
semantic highlighting. The phases follow a dependency chain: Phase 1 (static
|
||||
analysis) builds only on the AST, Phase 2 (the standalone assembler) emits
|
||||
object code, and Phases 3 (dynamic analysis) and 4 (the debugger) both consume
|
||||
the execution substrate that the assembler provides.
|
||||
|
||||
## Components
|
||||
## Packages
|
||||
|
||||
| Package | Responsibility |
|
||||
|---|---|
|
||||
| `token` | token kinds and positions |
|
||||
| `lexer` | hand-written scanner; permissive, and it never panics |
|
||||
| `ast` | the typed syntax tree: declarations, lines, operands |
|
||||
| `parser` | line-oriented parser producing the AST and its diagnostics |
|
||||
| `arch` | register and instruction tables for the four architectures |
|
||||
| `lint` | static checks over the AST |
|
||||
| `format` | canonical formatter over the token stream |
|
||||
| `lsp` | the language server |
|
||||
| `asm` | standalone assembler: encoders, image layout, object emitters |
|
||||
| `verify` | JIT execution, ABI checks, differential fuzzing |
|
||||
| `debug` | interactive ptrace debugger |
|
||||
| `cmd/gasm` | the CLI |
|
||||
| `_gen` | rebuilds the `arch` tables from the Go toolchain source |
|
||||
|
||||
The boundaries matter as much as the responsibilities: `ast` records syntax
|
||||
only, and whether a name is a register or a label is left to `arch`, so the
|
||||
parser stays architecture-agnostic. `asm` and `verify` are the only packages
|
||||
that touch machine code and executable memory, and `cmd/gasm` owns no logic
|
||||
beyond flags and output.
|
||||
|
||||
### `token` and `lexer`
|
||||
|
||||
@@ -206,7 +233,7 @@ The standalone assembler (Phase 2). Its core is an amd64 instruction encoder:
|
||||
a REX/ModR-M/SIB/displacement/immediate engine plus the scalar instruction set,
|
||||
with the Plan 9 operand order (source first) mapped onto the x86 encoding.
|
||||
Every encoding is validated by decoding it again with `golang.org/x/arch`, the
|
||||
one module dependency, used in tests only and never linked into the binary.
|
||||
one module dependency, which also backs the `gasm dis` listings.
|
||||
|
||||
A **RISC-V encoder** (Phase 5, RV64IMAFDC + RVC compression) encodes the full
|
||||
integer, atomic, float/double, FMA and CSR instruction sets with the MOV
|
||||
@@ -383,8 +410,8 @@ the Go ABI fixes across calls (amd64 `BP`/`R14`, arm64 `R29`/`R28`, riscv64
|
||||
raw return trampoline `leaveJITCheckedRaw` verifies them, restoring the
|
||||
saved registers before Go code resumes. riscv64 is validated end to
|
||||
end under qemu-user emulation; arm64 shares the same stack convention and
|
||||
fix; loong64 stays ground-truth-only until hardware validation (see
|
||||
docs/DECISIONS.md). `gasm verify` runs the JIT checks when the host
|
||||
fix; loong64 stays ground-truth-only until hardware validation.
|
||||
`gasm verify` runs the JIT checks when the host
|
||||
matches the kernel's architecture and the toolchain comparisons
|
||||
elsewhere.
|
||||
|
||||
@@ -436,14 +463,72 @@ watchdog is armed before the ptrace attach, so a sandboxed debuggee cannot
|
||||
block it), and `--cover` runs to completion with a breakpoint on every
|
||||
label and reports which blocks executed.
|
||||
|
||||
## Extension points
|
||||
### Extending the toolkit
|
||||
|
||||
- **New architecture:** add an entry to the generator in `_gen`, run
|
||||
`just gen`, and add a `buildXXX()` register file plus a case in `ForArch`.
|
||||
- **New lint rule:** add a function in `lint` and a rule-code constant.
|
||||
- **New LSP feature:** add a method case in `dispatch` and a handler.
|
||||
|
||||
The phases follow a dependency chain. Phase 1 (static analysis) builds only on
|
||||
the AST; Phase 2 (the standalone assembler) emits object code; Phases 3
|
||||
(dynamic analysis) and 4 (the debugger) both consume the execution substrate
|
||||
that the assembler provides.
|
||||
## Data flow
|
||||
|
||||
The main operation, assembling one file:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant CLI as gasm CLI
|
||||
participant Parser as parser
|
||||
participant Asm as asm
|
||||
participant Go as go toolchain
|
||||
User->>CLI: gasm asm --format goobj -p pkg -o k.o k_amd64.s
|
||||
CLI->>Parser: Parse(path, src)
|
||||
Parser-->>CLI: AST, diagnostics
|
||||
CLI->>Asm: AssembleFile(AST)
|
||||
Asm->>Asm: encode operands, settle label offsets, lay out data
|
||||
Asm-->>CLI: Image, code and data and relocations
|
||||
CLI->>Asm: GOObject(pkg, path)
|
||||
Asm->>Go: go list -json -export, externals only
|
||||
Go-->>Asm: package and symbol indices
|
||||
Asm-->>CLI: Go object bytes
|
||||
CLI-->>User: wrote N bytes to k.o
|
||||
```
|
||||
|
||||
Errors are produced where the parse or the encoding fails and become values at
|
||||
the CLI boundary: the parser returns a diagnostic list and never aborts a file,
|
||||
`AssembleFile` returns an error, and `cmd/gasm` prints what it has to stderr
|
||||
and returns a non-zero exit code. The formatter and the linter take the same
|
||||
AST by a different route: `gasm fmt` re-spaces the token stream and `gasm lint`
|
||||
walks the parsed file, so neither depends on an encoding.
|
||||
|
||||
## State and lifetime
|
||||
|
||||
- The analysis packages (`lexer`, `parser`, `format`, `lint`, `arch`) hold only
|
||||
read-only lookup tables and no mutable state: every call allocates its own
|
||||
tokens and AST, and any number of goroutines may read the `arch` tables.
|
||||
- A `verify.Kernel` owns one executable mapping, which `Close` releases. The
|
||||
JIT trampolines keep the Go stack pointer and the checked-call sentinels in
|
||||
package globals, so a call is a process-wide, one-at-a-time operation. The
|
||||
`gasm verify` sweeps therefore run each function in a child process, which
|
||||
contains a crash and keeps the globals unshared.
|
||||
- `lsp.Server` is long-lived: it runs a single read and dispatch loop over the
|
||||
stream and touches its document store only from that loop, so one server
|
||||
serves one connection.
|
||||
- A `debug.Session` owns a traced child process and pins its goroutine to the
|
||||
forking OS thread, because ptrace requests must stay on that thread.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **`golang.org/x/arch`** (v0.30.0) is the one module dependency: it is the
|
||||
disassembler backend (`gasm dis` and the debugger's listings) and the source
|
||||
of the register metadata the encoder consults (`asm/reg.go`, `asm/vex.go`).
|
||||
The tests additionally decode through it to validate the encodings.
|
||||
- **The Go toolchain**, as an oracle and never as a library: `go tool asm`
|
||||
supplies the object preamble and the ground truth for `gasm verify
|
||||
--ground-truth`, `go list -json -export` locates the archives of the packages
|
||||
a GOOBJ object references, and `_gen` parses
|
||||
`$GOROOT/src/cmd/internal/obj/<arch>/anames.go` to rebuild the tables.
|
||||
- **Linux process interfaces** for the dynamic work: `mmap` and `mprotect` for
|
||||
the JIT mapping, ptrace with `/proc/pid/mem` for the debugger. That is why
|
||||
`verify` runs a JIT check only when the host architecture matches the
|
||||
kernel's, and why `debug` is Linux-only.
|
||||
|
||||
Reference in New Issue
Block a user