docs: add the benchmarking document
Test / test (push) Canceled after 1m19s

Assisted-by: GLM 5.3 Flash
This commit is contained in:
2026-09-18 00:20:42 +02:00
parent 8aa2b1b9c0
commit 17574a0d15
2 changed files with 47 additions and 1 deletions
+44
View File
@@ -0,0 +1,44 @@
# Benchmarking
How the performance numbers attached to this project are measured, so that a
number in a changelog entry or a release note can be reproduced and trusted.
## The suite
The benchmarks live in `bench_test.go`, next to the code they measure:
| Benchmark | What it measures |
|---|---|
| `BenchmarkParse` | `Parse` over a representative configuration document |
| `BenchmarkMarshal` | `Marshal` of the tree `Parse` produced from the same document |
| `BenchmarkStrictDecode` | `Decode` into a struct under `DisallowUnknownFields` |
| `BenchmarkParseLong` | `Parse` over a generated document with about 2000 array-of-tables entries |
## Running
```sh
just bench
```
The recipe runs the suite with `-benchmem -count=5`. Every benchmark uses
`b.Loop`, so setup runs outside the timed region, and `ReportAllocs` records
allocations per operation. The parse and marshal benchmarks set `SetBytes`, so
their results read as input bytes per second.
## Method
- An idle machine only: a loaded box times whatever else is running, and the
fastest sample can land on the wrong function.
- An A/B comparison runs both variants inside one process, in one binary;
separate processes of identical binaries differ by more than the effect
being measured.
- The five counts are compared through their medians, allocations and bytes
per operation alongside the times. Differences within 1 to 2 percent are
noise; only a difference beyond that is a result.
- When timing is hopeless, the allocation and byte counts are the result.
## Reports
The repository stores no benchmark reports. A performance claim in
`CHANGELOG.md` is measured with the method above on the change that makes it,
and the number travels with the claim.
+3 -1
View File
@@ -77,7 +77,9 @@ just bench
``` ```
Benchmark on an idle machine, and compare only runs made in one process against Benchmark on an idle machine, and compare only runs made in one process against
each other. The recipe sweeps `./...` five times with `-benchmem`. each other. The recipe sweeps `./...` five times with `-benchmem`. The binding
measurement method, and what counts as a result, is in
[docs/BENCHMARKING.md](BENCHMARKING.md).
## Debugging the build ## Debugging the build