Assisted-by: GLM 5.3 Flash
This commit is contained in:
@@ -0,0 +1,44 @@
|
||||
# Benchmarking
|
||||
|
||||
How the performance numbers attached to this project are measured, so that a
|
||||
number in a changelog entry or a release note can be reproduced and trusted.
|
||||
|
||||
## The suite
|
||||
|
||||
The benchmarks live in `bench_test.go`, next to the code they measure:
|
||||
|
||||
| Benchmark | What it measures |
|
||||
|---|---|
|
||||
| `BenchmarkParse` | `Parse` over a representative configuration document |
|
||||
| `BenchmarkMarshal` | `Marshal` of the tree `Parse` produced from the same document |
|
||||
| `BenchmarkStrictDecode` | `Decode` into a struct under `DisallowUnknownFields` |
|
||||
| `BenchmarkParseLong` | `Parse` over a generated document with about 2000 array-of-tables entries |
|
||||
|
||||
## Running
|
||||
|
||||
```sh
|
||||
just bench
|
||||
```
|
||||
|
||||
The recipe runs the suite with `-benchmem -count=5`. Every benchmark uses
|
||||
`b.Loop`, so setup runs outside the timed region, and `ReportAllocs` records
|
||||
allocations per operation. The parse and marshal benchmarks set `SetBytes`, so
|
||||
their results read as input bytes per second.
|
||||
|
||||
## Method
|
||||
|
||||
- An idle machine only: a loaded box times whatever else is running, and the
|
||||
fastest sample can land on the wrong function.
|
||||
- An A/B comparison runs both variants inside one process, in one binary;
|
||||
separate processes of identical binaries differ by more than the effect
|
||||
being measured.
|
||||
- The five counts are compared through their medians, allocations and bytes
|
||||
per operation alongside the times. Differences within 1 to 2 percent are
|
||||
noise; only a difference beyond that is a result.
|
||||
- When timing is hopeless, the allocation and byte counts are the result.
|
||||
|
||||
## Reports
|
||||
|
||||
The repository stores no benchmark reports. A performance claim in
|
||||
`CHANGELOG.md` is measured with the method above on the change that makes it,
|
||||
and the number travels with the claim.
|
||||
+3
-1
@@ -77,7 +77,9 @@ just bench
|
||||
```
|
||||
|
||||
Benchmark on an idle machine, and compare only runs made in one process against
|
||||
each other. The recipe sweeps `./...` five times with `-benchmem`.
|
||||
each other. The recipe sweeps `./...` five times with `-benchmem`. The binding
|
||||
measurement method, and what counts as a result, is in
|
||||
[docs/BENCHMARKING.md](BENCHMARKING.md).
|
||||
|
||||
## Debugging the build
|
||||
|
||||
|
||||
Reference in New Issue
Block a user