Assisted-by: GLM 5.3 Flash
This commit is contained in:
@@ -0,0 +1,44 @@
|
|||||||
|
# Benchmarking
|
||||||
|
|
||||||
|
How the performance numbers attached to this project are measured, so that a
|
||||||
|
number in a changelog entry or a release note can be reproduced and trusted.
|
||||||
|
|
||||||
|
## The suite
|
||||||
|
|
||||||
|
The benchmarks live in `bench_test.go`, next to the code they measure:
|
||||||
|
|
||||||
|
| Benchmark | What it measures |
|
||||||
|
|---|---|
|
||||||
|
| `BenchmarkParse` | `Parse` over a representative configuration document |
|
||||||
|
| `BenchmarkMarshal` | `Marshal` of the tree `Parse` produced from the same document |
|
||||||
|
| `BenchmarkStrictDecode` | `Decode` into a struct under `DisallowUnknownFields` |
|
||||||
|
| `BenchmarkParseLong` | `Parse` over a generated document with about 2000 array-of-tables entries |
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```sh
|
||||||
|
just bench
|
||||||
|
```
|
||||||
|
|
||||||
|
The recipe runs the suite with `-benchmem -count=5`. Every benchmark uses
|
||||||
|
`b.Loop`, so setup runs outside the timed region, and `ReportAllocs` records
|
||||||
|
allocations per operation. The parse and marshal benchmarks set `SetBytes`, so
|
||||||
|
their results read as input bytes per second.
|
||||||
|
|
||||||
|
## Method
|
||||||
|
|
||||||
|
- An idle machine only: a loaded box times whatever else is running, and the
|
||||||
|
fastest sample can land on the wrong function.
|
||||||
|
- An A/B comparison runs both variants inside one process, in one binary;
|
||||||
|
separate processes of identical binaries differ by more than the effect
|
||||||
|
being measured.
|
||||||
|
- The five counts are compared through their medians, allocations and bytes
|
||||||
|
per operation alongside the times. Differences within 1 to 2 percent are
|
||||||
|
noise; only a difference beyond that is a result.
|
||||||
|
- When timing is hopeless, the allocation and byte counts are the result.
|
||||||
|
|
||||||
|
## Reports
|
||||||
|
|
||||||
|
The repository stores no benchmark reports. A performance claim in
|
||||||
|
`CHANGELOG.md` is measured with the method above on the change that makes it,
|
||||||
|
and the number travels with the claim.
|
||||||
+3
-1
@@ -77,7 +77,9 @@ just bench
|
|||||||
```
|
```
|
||||||
|
|
||||||
Benchmark on an idle machine, and compare only runs made in one process against
|
Benchmark on an idle machine, and compare only runs made in one process against
|
||||||
each other. The recipe sweeps `./...` five times with `-benchmem`.
|
each other. The recipe sweeps `./...` five times with `-benchmem`. The binding
|
||||||
|
measurement method, and what counts as a result, is in
|
||||||
|
[docs/BENCHMARKING.md](BENCHMARKING.md).
|
||||||
|
|
||||||
## Debugging the build
|
## Debugging the build
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user