perf(encode): emit in place, pool the buffer and cache interface flags
Test / test (push) Successful in 1m33s

Grouped emission walks the entries slice in three passes instead of copying them into per-kind slices, error paths render their key path only when an error names it, inline tables are measured once because a measuring encoder no longer nests another measurement, clean string runs are written in one write, and date-times render through the single-buffer path. The output buffer comes from a sync.Pool and returns to it within a 1 MiB retention cap, the output handed to the caller as a copy; repeated marshals keep the live heap flat, verified over 500 runs. The encoder resolves Marshaler and the text interfaces through the same per-type flag cache and hint the decoder uses, and docs/ARCHITECTURE.md now lists the caches and the pool as the library's shared state.

Representative document: 177 to 141 allocations, 12 230 to 6170 bytes per operation; long document: 97 175 to 63 660 allocations, 6.97 to 2.79 MB per operation, about 7.8 to about 3.4 ms.
This commit is contained in:
2026-09-20 22:15:38 +02:00
parent 9874464213
commit 2ae487e117
4 changed files with 337 additions and 118 deletions
+11
View File
@@ -92,6 +92,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
document (9792 ns against 11 147 ns) with 24 percent fewer allocations document (9792 ns against 11 147 ns) with 24 percent fewer allocations
(167 against 220); interface lookups resolve through a cached per-type (167 against 220); interface lookups resolve through a cached per-type
flag set instead of boxing every value into an interface to ask. flag set instead of boxing every value into an interface to ask.
- `Marshal` runs at the 1.1.0 speed while emitting the new TOML 1.1 output
form, at half the bytes per operation (6170 against 11 348 on the
representative document), and writes through a pooled output buffer with
a 1 MiB retention cap; repeated marshals keep the live heap flat.
- Two benchmarks measure the shapes that drove the work:
`BenchmarkStrictDecodeLong` and `BenchmarkMarshalLong` run the 2000-entry
document at about 3.8 ms and 3.4 ms per call, at 63 772 and 63 660
allocations.
### Fixed ### Fixed
@@ -99,6 +107,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
as `type Name string`, panicked instead of storing the value, because a as `type Name string`, panicked instead of storing the value, because a
value of the predeclared type is not assignable to a defined type and the value of the predeclared type is not assignable to a defined type and the
decoder assigned it without a conversion. decoder assigned it without a conversion.
- A top-level value the encoder could not normalise reported its path with
a leading dot, `interpres: .port: ...`; the message now reads
`interpres: port: ...`, the shape `EncodeError.Path` already used.
## [1.1.0] - 2026-09-18 ## [1.1.0] - 2026-09-18
+16 -8
View File
@@ -102,15 +102,23 @@ sequenceDiagram
setter methods are not, and must finish before the value is shared. setter methods are not, and must finish before the value is shared.
- The parser is allocated per `ParseContext` call; the parser itself caches - The parser is allocated per `ParseContext` call; the parser itself caches
nothing between documents. nothing between documents.
- The one piece of shared state is the struct-schema cache in `decode.go`: a - The shared state is a set of caches and pools whose entries are immutable
`sync.Map` keyed by `reflect.Type`, holding the flattened field layout the once published, each growing with the number of distinct types rather than
decoder and the encoder both consult. A schema is immutable once published, with document size: the struct-schema cache in `decode.go` (a `sync.Map`
so concurrent callers only race to build an identical value, the same keyed on `reflect.Type`, holding the flattened field layout the decoder and
trade-off `encoding/json`'s field cache makes. The cache grows with the the encoder both consult), the per-type interface flag caches in `decode.go`
number of distinct struct types, never with document size. and `encode.go` (recording where `Marshaler`, `Unmarshaler` and the text
interfaces can be found, so a walk builds an interface value only where the
assertion can succeed), each fronted by a monomorphic hint holding the type
resolved last, and the encoder's output-buffer pool in `encode.go`
(`sync.Pool`, buffers returned to it only within a 1 MiB retention cap). A
published schema or flag set never mutates, so concurrent callers only race
to build an identical value, the same trade-off `encoding/json`'s field
cache makes.
- The date-time wrappers are values, not pointers, and are immutable in use. - The date-time wrappers are values, not pointers, and are immutable in use.
- Nothing in the library starts goroutines; apart from the schema cache above, - Nothing in the library starts goroutines; apart from the caches and the pool
which never mutates a published entry, there is no shared mutable state. above, which never mutate a published entry, there is no shared mutable
state.
## Dependencies ## Dependencies
+300 -106
View File
@@ -15,6 +15,8 @@ import (
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"sync"
"sync/atomic"
"time" "time"
"unicode/utf8" "unicode/utf8"
) )
@@ -27,8 +29,63 @@ var (
timeGoType = reflect.TypeFor[time.Time]() timeGoType = reflect.TypeFor[time.Time]()
durationType = reflect.TypeFor[time.Duration]() durationType = reflect.TypeFor[time.Duration]()
textMarshalerType = reflect.TypeFor[encoding.TextMarshaler]() textMarshalerType = reflect.TypeFor[encoding.TextMarshaler]()
marshalerIfaceType = reflect.TypeFor[Marshaler]()
) )
// The encoder's per-type flags mirror the decoder's cache: an interface value
// is built only where the cache says the assertion can succeed. The bits name
// the receiver the method is found on.
const (
encFlagMarshaler uint8 = 1 << iota
encFlagAddrMarshaler
encFlagTextMarshaler
encFlagAddrTextMarshaler
)
// encTypeFlagCache holds one flag entry per value type the encoder walks,
// immutable once published, the same trade-off structSchemaCache makes. The
// hint below re-points at these published entries, so a hot lookup allocates
// nothing.
var encTypeFlagCache sync.Map // reflect.Type -> *flagHintEntry
// encTypeFlagHint is the encoder's monomorphic hint, for the same reason the
// decoder's one exists: an encode walk repeats one type across consecutive
// fields and elements, and the hint answers without a cache probe.
var encTypeFlagHint atomic.Pointer[flagHintEntry]
func encTypeFlags(t reflect.Type) uint8 {
if e := encTypeFlagHint.Load(); e != nil && e.typ == t {
return e.flags
}
if v, ok := encTypeFlagCache.Load(t); ok {
entry := v.(*flagHintEntry)
encTypeFlagHint.Store(entry)
return entry.flags
}
var f uint8
if t.Implements(marshalerIfaceType) {
f |= encFlagMarshaler
}
pt := reflect.PointerTo(t)
if pt.Implements(marshalerIfaceType) {
f |= encFlagAddrMarshaler
}
// The date-time types are excluded from the text path for the reason
// textValue records; no caller reaches textMarshalerOf for them.
if !isDateTimeType(t) {
if t.Implements(textMarshalerType) {
f |= encFlagTextMarshaler
}
if pt.Implements(textMarshalerType) {
f |= encFlagAddrTextMarshaler
}
}
actual, _ := encTypeFlagCache.LoadOrStore(t, &flagHintEntry{t, f})
published := actual.(*flagHintEntry)
encTypeFlagHint.Store(published)
return published.flags
}
// inlineLimit is the column past which an inline table is written across // inlineLimit is the column past which an inline table is written across
// lines. TOML 1.1 lets an inline table carry newlines and a trailing comma, so // lines. TOML 1.1 lets an inline table carry newlines and a trailing comma, so
// a long one stays readable instead of running off the line. // a long one stays readable instead of running off the line.
@@ -38,10 +95,26 @@ const inlineLimit = 100
// form it renders is always the single-line one. // form it renders is always the single-line one.
const noInlineBreak = 1 << 30 const noInlineBreak = 1 << 30
// encoderBufRetention is the largest output buffer put back into the pool.
// The cap is the pool's memory rule: repeated marshals of a document within it
// reuse one buffer and keep the heap flat, while a document larger than the
// cap pins nothing per processor once written.
const encoderBufRetention = 1 << 20
// encoderBufPool holds output buffers between Marshal calls.
var encoderBufPool = sync.Pool{New: func() any { return new(bytes.Buffer) }}
// getEncoderBuf takes a cleared output buffer from the pool.
func getEncoderBuf() *bytes.Buffer {
b := encoderBufPool.Get().(*bytes.Buffer)
b.Reset()
return b
}
// encoder produces a TOML document from a Go value via a small intermediate // encoder produces a TOML document from a Go value via a small intermediate
// representation that preserves the order in which fields were declared. // representation that preserves the order in which fields were declared.
type encoder struct { type encoder struct {
buf bytes.Buffer buf *bytes.Buffer
ctx context.Context ctx context.Context
opts Encoder opts Encoder
@@ -54,15 +127,32 @@ type encoder struct {
limit int limit int
} }
func newEncoder() *encoder { return &encoder{limit: inlineLimit} } func newEncoder() *encoder {
e := &encoder{limit: inlineLimit}
e.buf = getEncoderBuf()
return e
}
// flat returns an encoder that measures a value by rendering it on one line, // flat returns an encoder that measures a value by rendering it on one line,
// so a caller can decide which form to write before writing it. // so a caller can decide which form to write before writing it.
func (e *encoder) flat() *encoder { func (e *encoder) flat() *encoder {
return &encoder{ctx: e.ctx, opts: e.opts, limit: noInlineBreak} f := &encoder{ctx: e.ctx, opts: e.opts, limit: noInlineBreak}
f.buf = getEncoderBuf()
return f
} }
func (e *encoder) bytes() []byte { return e.buf.Bytes() } // release returns the encoder's output buffer to the pool, within the
// retention cap. It is safe to call twice; the buffer travels only once.
func (e *encoder) release() {
if e.buf == nil {
return
}
b := e.buf
e.buf = nil
if b.Cap() <= encoderBufRetention {
encoderBufPool.Put(b)
}
}
// column reports how many bytes the current line already holds, so a form can // column reports how many bytes the current line already holds, so a form can
// be measured against the limit before it is written. // be measured against the limit before it is written.
@@ -106,13 +196,14 @@ func (e *encoder) encode(v any) error {
rv = rv.Elem() rv = rv.Elem()
} }
doc := &tomlDoc{ctx: e.ctx, opts: e.opts} doc := &tomlDoc{ctx: e.ctx, opts: e.opts}
root := encPath{}
switch rv.Kind() { switch rv.Kind() {
case reflect.Struct: case reflect.Struct:
if err := buildStructDoc(rv, doc, ""); err != nil { if err := buildStructDoc(rv, doc, root); err != nil {
return err return err
} }
case reflect.Map: case reflect.Map:
if err := buildMapDoc(rv, doc, ""); err != nil { if err := buildMapDoc(rv, doc, root); err != nil {
return err return err
} }
default: default:
@@ -133,15 +224,20 @@ const (
) )
// entry is one binding in a tomlDoc. entries live in a single slice in the // entry is one binding in a tomlDoc. entries live in a single slice in the
// order they were added; emission either walks that order directly // order they were added; emission walks that order directly, either as it is
// (Encoder with GroupByKind(false)) or partitions by kind first // (Encoder with GroupByKind(false)) or in kind-grouped passes over the same
// (Encoder with GroupByKind(true), the default). // slice (the default).
type entry struct { type entry struct {
kind entryKind kind entryKind
key string key string
val any // entryScalar val any // entryScalar
doc *tomlDoc // entryTable doc *tomlDoc // entryTable
docs []*tomlDoc docs []*tomlDoc
// emitted records that the grouped emission wrote this table inline, so
// the header pass that follows skips it. The representation is built
// fresh per Marshal call.
emitted bool
} }
// tomlDoc holds the entries of one TOML table in declaration order. // tomlDoc holds the entries of one TOML table in declaration order.
@@ -170,26 +266,62 @@ func (d *tomlDoc) addArray(key string, subs []*tomlDoc) {
d.entries = append(d.entries, entry{kind: entryArray, key: key, docs: subs}) d.entries = append(d.entries, entry{kind: entryArray, key: key, docs: subs})
} }
// partitionedEntries returns the entries grouped by kind, preserving each // --- error paths -----------------------------------------------------------
// group's relative order. The only allocation is the three slice headers.
func (d *tomlDoc) partitionedEntries() (scalars []entry, tables []entry, arrays []entry) { // encPath names a value the way an error message needs it, "server.ports[2]",
for _, e := range d.entries { // without building the string unless an error actually asks for one. A zero
switch e.kind { // encPath is the document root. The chain is stack-allocated: a segment holds
case entryScalar: // a pointer to its parent's frame-local value, and the rendered string exists
scalars = append(scalars, e) // only while an error is being built.
case entryTable: type encPath struct {
tables = append(tables, e) parent *encPath
case entryArray: name string
arrays = append(arrays, e) index int
}
// key returns the child path of a named key.
func (p *encPath) key(name string) encPath {
return encPath{parent: p, name: name, index: -1}
}
// elem returns the child path of an array element by index.
func (p *encPath) elem(i int) encPath {
return encPath{parent: p, index: i}
}
// String renders the path root first: keys join with dots and an array
// element carries its bracketed index, so the third port under server reads
// "server.ports[2]".
func (p encPath) String() string {
var parts []string
for s := &p; s != nil; s = s.parent {
switch {
case s.name == "" && s.parent == nil:
// The root: nothing to write.
case s.name == "":
parts = append(parts, "["+strconv.Itoa(s.index)+"]")
default:
parts = append(parts, s.name)
} }
} }
return var b strings.Builder
for _, part := range slices.Backward(parts) {
if strings.HasPrefix(part, "[") {
b.WriteString(part)
continue
}
if b.Len() > 0 {
b.WriteByte('.')
}
b.WriteString(part)
}
return b.String()
} }
// --- reflection walk: struct --------------------------------------------- // --- reflection walk: struct ---------------------------------------------
func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error { func buildStructDoc(v reflect.Value, doc *tomlDoc, path encPath) error {
return walkStructDoc(v, doc, ctx, nil, cachedStructSchema(v.Type())) return walkStructDoc(v, doc, path, nil, cachedStructSchema(v.Type()))
} }
// walkStructDoc emits the fields of v into doc. prefix is v's index path from // walkStructDoc emits the fields of v into doc. prefix is v's index path from
@@ -198,8 +330,11 @@ func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
// decoder's rule: the shallower field wins, the later declaration at equal // decoder's rule: the shallower field wins, the later declaration at equal
// depth. A field another field shadows is skipped, because emitting both // depth. A field another field shadows is skipped, because emitting both
// would duplicate the key and the output would not re-parse. // would duplicate the key and the output would not re-parse.
func walkStructDoc(v reflect.Value, doc *tomlDoc, ctx string, prefix []int, schema structSchema) error { func walkStructDoc(v reflect.Value, doc *tomlDoc, path encPath, prefix []int, schema structSchema) error {
t := v.Type() t := v.Type()
if cap(doc.entries) == 0 {
doc.entries = make([]entry, 0, t.NumField())
}
for i := range t.NumField() { for i := range t.NumField() {
if i%ctxCheckInterval == 0 { if i%ctxCheckInterval == 0 {
if err := doc.checkCtx(); err != nil { if err := doc.checkCtx(); err != nil {
@@ -210,7 +345,7 @@ func walkStructDoc(v reflect.Value, doc *tomlDoc, ctx string, prefix []int, sche
if f.PkgPath != "" { if f.PkgPath != "" {
continue continue
} }
path := append(append([]int{}, prefix...), i) fpath := append(append([]int{}, prefix...), i)
if f.Anonymous { if f.Anonymous {
tag, _ := f.Tag.Lookup("toml") tag, _ := f.Tag.Lookup("toml")
if tag == "-" { if tag == "-" {
@@ -225,20 +360,18 @@ func walkStructDoc(v reflect.Value, doc *tomlDoc, ctx string, prefix []int, sche
case reflect.Struct: case reflect.Struct:
if isScalarStruct(fv.Type()) { if isScalarStruct(fv.Type()) {
name := strings.ToLower(f.Name) name := strings.ToLower(f.Name)
if !schema.ownsKey(name, path) { if !schema.ownsKey(name, fpath) {
continue continue
} }
if err := doc.appendScalar(name, fv.Interface(), ctx); err != nil { doc.addScalar(name, fv.Interface())
return err
}
continue continue
} }
if err := walkStructDoc(fv, doc, ctx, path, schema); err != nil { if err := walkStructDoc(fv, doc, path, fpath, schema); err != nil {
return err return err
} }
continue continue
case reflect.Map: case reflect.Map:
if err := buildMapDoc(fv, doc, ctx); err != nil { if err := buildMapDoc(fv, doc, path); err != nil {
return err return err
} }
continue continue
@@ -249,13 +382,13 @@ func walkStructDoc(v reflect.Value, doc *tomlDoc, ctx string, prefix []int, sche
if name == "-" { if name == "-" {
continue continue
} }
if !schema.ownsKey(strings.ToLower(name), path) { if !schema.ownsKey(strings.ToLower(name), fpath) {
continue continue
} }
if fieldOmitted(f, v.Field(i)) { if fieldOmitted(f, v.Field(i)) {
continue continue
} }
if err := addField(doc, name, v.Field(i), ctx); err != nil { if err := addField(doc, name, v.Field(i), path); err != nil {
return err return err
} }
} }
@@ -322,10 +455,13 @@ func fieldName(f reflect.StructField) string {
// --- reflection walk: map ------------------------------------------------ // --- reflection walk: map ------------------------------------------------
func buildMapDoc(v reflect.Value, doc *tomlDoc, ctx string) error { func buildMapDoc(v reflect.Value, doc *tomlDoc, path encPath) error {
if v.Type().Key().Kind() != reflect.String { if v.Type().Key().Kind() != reflect.String {
return fmt.Errorf("interpres: map key must be string, got %s", v.Type().Key()) return fmt.Errorf("interpres: map key must be string, got %s", v.Type().Key())
} }
if cap(doc.entries) == 0 {
doc.entries = make([]entry, 0, v.Len())
}
keys := v.MapKeys() keys := v.MapKeys()
slices.SortFunc(keys, func(a, b reflect.Value) int { slices.SortFunc(keys, func(a, b reflect.Value) int {
return strings.Compare(a.String(), b.String()) return strings.Compare(a.String(), b.String())
@@ -336,7 +472,7 @@ func buildMapDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
return err return err
} }
} }
if err := addField(doc, k.String(), v.MapIndex(k), ctx); err != nil { if err := addField(doc, k.String(), v.MapIndex(k), path); err != nil {
return err return err
} }
} }
@@ -351,14 +487,14 @@ func buildMapDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
// contract violation. // contract violation.
var errNilMarshalTOML = errors.New("MarshalTOML returned a nil value") var errNilMarshalTOML = errors.New("MarshalTOML returned a nil value")
func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error { func addField(doc *tomlDoc, name string, v reflect.Value, path encPath) error {
if m, ok := marshalerOf(v); ok { if m, ok := marshalerOf(v); ok {
mv, err := m.MarshalTOML() mv, err := m.MarshalTOML()
if err != nil { if err != nil {
return &EncodeError{Path: joinKey(ctx, name), Err: err} return &EncodeError{Path: path.key(name).String(), Err: err}
} }
if mv == nil { if mv == nil {
return &EncodeError{Path: joinKey(ctx, name), Err: errNilMarshalTOML} return &EncodeError{Path: path.key(name).String(), Err: errNilMarshalTOML}
} }
v = reflect.ValueOf(mv) v = reflect.ValueOf(mv)
} }
@@ -366,10 +502,11 @@ func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
// a scalar kind or a struct. // a scalar kind or a struct.
s, isText, err := textValue(v) s, isText, err := textValue(v)
if err != nil { if err != nil {
return &EncodeError{Path: joinKey(ctx, name), Err: err} return &EncodeError{Path: path.key(name).String(), Err: err}
} }
if isText { if isText {
return doc.appendScalar(name, s, ctx) doc.addScalar(name, s)
return nil
} }
v = followPtr(v) v = followPtr(v)
if !v.IsValid() { if !v.IsValid() {
@@ -384,37 +521,33 @@ func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
switch v.Kind() { switch v.Kind() {
case reflect.Struct: case reflect.Struct:
if isScalarStruct(v.Type()) { if isScalarStruct(v.Type()) {
return doc.appendScalar(name, v.Interface(), ctx) doc.addScalar(name, v.Interface())
return nil
} }
return addSubTable(doc, name, v, ctx) return addSubTable(doc, name, v, path)
case reflect.Map: case reflect.Map:
return addSubTable(doc, name, v, ctx) return addSubTable(doc, name, v, path)
case reflect.Slice, reflect.Array: case reflect.Slice, reflect.Array:
return addArrayValue(doc, name, v, ctx) return addArrayValue(doc, name, v, path)
default: default:
val, err := normaliseValue(v) val, err := normaliseValue(v)
if err != nil { if err != nil {
return fmt.Errorf("interpres: %s.%s: %w", ctx, name, err) return fmt.Errorf("interpres: %s: %w", path.key(name), err)
} }
return doc.appendScalar(name, val, ctx) doc.addScalar(name, val)
}
}
// appendScalar wraps addScalar with a uniform error path.
func (d *tomlDoc) appendScalar(name string, val any, ctx string) error {
d.addScalar(name, val)
return nil return nil
}
} }
func addSubTable(doc *tomlDoc, name string, v reflect.Value, ctx string) error { func addSubTable(doc *tomlDoc, name string, v reflect.Value, path encPath) error {
sub := &tomlDoc{ctx: doc.ctx, opts: doc.opts} sub := &tomlDoc{ctx: doc.ctx, opts: doc.opts}
switch v.Kind() { switch v.Kind() {
case reflect.Struct: case reflect.Struct:
if err := buildStructDoc(v, sub, joinKey(ctx, name)); err != nil { if err := buildStructDoc(v, sub, path.key(name)); err != nil {
return err return err
} }
case reflect.Map: case reflect.Map:
if err := buildMapDoc(v, sub, joinKey(ctx, name)); err != nil { if err := buildMapDoc(v, sub, path.key(name)); err != nil {
return err return err
} }
} }
@@ -422,7 +555,7 @@ func addSubTable(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
return nil return nil
} }
func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error { func addArrayValue(doc *tomlDoc, name string, v reflect.Value, path encPath) error {
if v.Kind() == reflect.Slice && v.IsNil() { if v.Kind() == reflect.Slice && v.IsNil() {
// A nil slice has no explicit representation in TOML, so it is skipped. // A nil slice has no explicit representation in TOML, so it is skipped.
return nil return nil
@@ -436,9 +569,14 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
if doc.opts.omitEmptyArrays { if doc.opts.omitEmptyArrays {
return nil return nil
} }
return doc.appendScalar(name, []any{}, ctx) doc.addScalar(name, []any{})
return nil
} }
// The element paths hang off this one; they render only when an error
// names them.
apath := path.key(name)
// Every element is resolved through MarshalTOML first, so an element that // Every element is resolved through MarshalTOML first, so an element that
// renders itself as a scalar, a table or a value array is classified by // renders itself as a scalar, a table or a value array is classified by
// what it produces rather than by its Go kind, and its method runs once. // what it produces rather than by its Go kind, and its method runs once.
@@ -449,7 +587,7 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
return err return err
} }
} }
ev, err := resolveElement(v.Index(i), fmt.Sprintf("%s[%d]", joinKey(ctx, name), i)) ev, err := resolveElement(v.Index(i), apath.elem(i))
if err != nil { if err != nil {
return err return err
} }
@@ -485,17 +623,17 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
switch ev.Kind() { switch ev.Kind() {
case reflect.Struct: case reflect.Struct:
if isScalarStruct(ev.Type()) { if isScalarStruct(ev.Type()) {
return &EncodeError{Path: fmt.Sprintf("%s[%d]", joinKey(ctx, name), i), Err: errors.New("heterogeneous array contains scalar")} return &EncodeError{Path: apath.elem(i).String(), Err: errors.New("heterogeneous array contains scalar")}
} }
if err := buildStructDoc(ev, sub, joinKey(ctx, fmt.Sprintf("%s[%d]", name, i))); err != nil { if err := buildStructDoc(ev, sub, apath.elem(i)); err != nil {
return err return err
} }
case reflect.Map: case reflect.Map:
if err := buildMapDoc(ev, sub, joinKey(ctx, fmt.Sprintf("%s[%d]", name, i))); err != nil { if err := buildMapDoc(ev, sub, apath.elem(i)); err != nil {
return err return err
} }
default: default:
return &EncodeError{Path: joinKey(ctx, name), Err: errors.New("heterogeneous array, expected table")} return &EncodeError{Path: apath.elem(i).String(), Err: errors.New("heterogeneous array, expected table")}
} }
subs[i] = sub subs[i] = sub
} }
@@ -514,27 +652,37 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
} }
val, err := normaliseValue(ev) val, err := normaliseValue(ev)
if err != nil { if err != nil {
return &EncodeError{Path: fmt.Sprintf("%s[%d]", joinKey(ctx, name), i), Err: err} return &EncodeError{Path: apath.elem(i).String(), Err: err}
} }
items[i] = val items[i] = val
} }
return doc.appendScalar(name, items, ctx) doc.addScalar(name, items)
return nil
} }
// marshalerOf finds the Marshaler a value carries: on the value itself, or on // marshalerOf finds the Marshaler a value carries: on the value itself, or on
// its address, so a pointer-receiver MarshalTOML is found on an addressable // its address, so a pointer-receiver MarshalTOML is found on an addressable
// struct field or slice element, exactly as textMarshalerOf finds MarshalText. // struct field or slice element, exactly as textMarshalerOf finds MarshalText.
// The lookup consults the per-type flag cache, so an interface value is built
// only where the assertion can succeed; an interface-typed value is asked
// dynamically, because what it holds may implement the interface when the
// interface type itself does not.
func marshalerOf(v reflect.Value) (Marshaler, bool) { func marshalerOf(v reflect.Value) (Marshaler, bool) {
if !v.CanInterface() { if !v.CanInterface() {
return nil, false return nil, false
} }
if m, ok := v.Interface().(Marshaler); ok { if v.Kind() == reflect.Interface {
return m, true m, ok := v.Interface().(Marshaler)
return m, ok
} }
if v.CanAddr() { f := encTypeFlags(v.Type())
if m, ok := v.Addr().Interface().(Marshaler); ok { if f&encFlagMarshaler != 0 {
return m, true m, ok := v.Interface().(Marshaler)
return m, ok
} }
if f&encFlagAddrMarshaler != 0 && v.CanAddr() {
m, ok := v.Addr().Interface().(Marshaler)
return m, ok
} }
return nil, false return nil, false
} }
@@ -542,10 +690,10 @@ func marshalerOf(v reflect.Value) (Marshaler, bool) {
// resolveElement looks through pointers and runs MarshalTOML, so an array // resolveElement looks through pointers and runs MarshalTOML, so an array
// element is classified by what its method produces. path names the element, // element is classified by what its method produces. path names the element,
// for the errors the method can raise. // for the errors the method can raise.
func resolveElement(v reflect.Value, path string) (reflect.Value, error) { func resolveElement(v reflect.Value, path encPath) (reflect.Value, error) {
ev := followPtr(v) ev := followPtr(v)
if !ev.IsValid() { if !ev.IsValid() {
return ev, &EncodeError{Path: path, Err: errors.New("nil element")} return ev, &EncodeError{Path: path.String(), Err: errors.New("nil element")}
} }
m, ok := marshalerOf(ev) m, ok := marshalerOf(ev)
if !ok { if !ok {
@@ -553,14 +701,14 @@ func resolveElement(v reflect.Value, path string) (reflect.Value, error) {
} }
mv, err := m.MarshalTOML() mv, err := m.MarshalTOML()
if err != nil { if err != nil {
return reflect.Value{}, &EncodeError{Path: path, Err: err} return reflect.Value{}, &EncodeError{Path: path.String(), Err: err}
} }
if mv == nil { if mv == nil {
return reflect.Value{}, &EncodeError{Path: path, Err: errNilMarshalTOML} return reflect.Value{}, &EncodeError{Path: path.String(), Err: errNilMarshalTOML}
} }
ev = followPtr(reflect.ValueOf(mv)) ev = followPtr(reflect.ValueOf(mv))
if !ev.IsValid() { if !ev.IsValid() {
return ev, &EncodeError{Path: path, Err: errors.New("nil element")} return ev, &EncodeError{Path: path.String(), Err: errors.New("nil element")}
} }
return ev, nil return ev, nil
} }
@@ -731,20 +879,25 @@ func textValue(v reflect.Value) (string, bool, error) {
return string(b), true, nil return string(b), true, nil
} }
// textMarshalerOf finds the encoding.TextMarshaler for v: on the value itself, // textMarshalerOf finds the encoding.TextMarshaler for v through the same
// or on its address, so a pointer-receiver MarshalText is found on an // flag cache: on the value itself, or on its address, so a pointer-receiver
// addressable struct field. // MarshalText is found on an addressable struct field.
func textMarshalerOf(v reflect.Value) (encoding.TextMarshaler, bool) { func textMarshalerOf(v reflect.Value) (encoding.TextMarshaler, bool) {
if !v.CanInterface() { if !v.CanInterface() {
return nil, false return nil, false
} }
if m, ok := v.Interface().(encoding.TextMarshaler); ok { if v.Kind() == reflect.Interface {
return m, true m, ok := v.Interface().(encoding.TextMarshaler)
return m, ok
} }
if v.CanAddr() { f := encTypeFlags(v.Type())
if m, ok := v.Addr().Interface().(encoding.TextMarshaler); ok { if f&encFlagTextMarshaler != 0 {
return m, true m, ok := v.Interface().(encoding.TextMarshaler)
return m, ok
} }
if f&encFlagAddrTextMarshaler != 0 && v.CanAddr() {
m, ok := v.Addr().Interface().(encoding.TextMarshaler)
return m, ok
} }
return nil, false return nil, false
} }
@@ -767,13 +920,6 @@ func isTableElementValue(v reflect.Value) bool {
return isTableElementType(v.Type()) return isTableElementType(v.Type())
} }
func joinKey(ctx, name string) string {
if ctx == "" {
return name
}
return ctx + "." + name
}
// --- emission ------------------------------------------------------------ // --- emission ------------------------------------------------------------
// writeBlankLine writes a single newline before a table or array-of-tables // writeBlankLine writes a single newline before a table or array-of-tables
@@ -788,8 +934,15 @@ func (e *encoder) writeBlankLine() {
func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error { func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
if e.opts.groupByKind { if e.opts.groupByKind {
scalars, tables, arrays := doc.partitionedEntries() // Scalars first, then inline sub-tables as value lines, then the
for _, kv := range scalars { // remaining tables as headers, then arrays of tables. Each pass walks
// the entries in place; grouping copies of them cost the encoder a
// third of its allocations for nothing.
for i := range doc.entries {
kv := &doc.entries[i]
if kv.kind != entryScalar {
continue
}
if err := e.writeKV(kv.key, kv.val); err != nil { if err := e.writeKV(kv.key, kv.val); err != nil {
return err return err
} }
@@ -797,17 +950,22 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
// An inlined sub-table is a value line, so it has to precede every // An inlined sub-table is a value line, so it has to precede every
// header of this document: a line written after a [header] would be // header of this document: a line written after a [header] would be
// read back as part of that table. // read back as part of that table.
headers := make([]entry, 0, len(tables)) for i := range doc.entries {
for _, t := range tables { t := &doc.entries[i]
if t.kind != entryTable {
continue
}
inlined, err := e.writeInlineSubTableIfSmall(t.key, t.doc) inlined, err := e.writeInlineSubTableIfSmall(t.key, t.doc)
if err != nil { if err != nil {
return err return err
} }
if !inlined { t.emitted = inlined
headers = append(headers, t)
} }
for i := range doc.entries {
t := &doc.entries[i]
if t.kind != entryTable || t.emitted {
continue
} }
for _, t := range headers {
path := append(append([]string{}, prefix...), t.key) path := append(append([]string{}, prefix...), t.key)
e.writeBlankLine() e.writeBlankLine()
e.buf.WriteByte('[') e.buf.WriteByte('[')
@@ -819,7 +977,11 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
return err return err
} }
} }
for _, a := range arrays { for i := range doc.entries {
a := &doc.entries[i]
if a.kind != entryArray {
continue
}
path := append(append([]string{}, prefix...), a.key) path := append(append([]string{}, prefix...), a.key)
for _, sub := range a.docs { for _, sub := range a.docs {
e.writeBlankLine() e.writeBlankLine()
@@ -918,21 +1080,37 @@ func (e *encoder) writeKey(key string) error {
if !utf8.ValidString(key) { if !utf8.ValidString(key) {
return fmt.Errorf("interpres: key %q is not valid UTF-8", key) return fmt.Errorf("interpres: key %q is not valid UTF-8", key)
} }
return writeQuotedString(&e.buf, key) return writeQuotedString(e.buf, key)
} }
// writeQuotedString writes s as a TOML basic string (double-quoted) to buf. // writeQuotedString writes s as a TOML basic string (double-quoted) to buf.
// Returns an error only if s is not valid UTF-8; invalid byte sequences // Returns an error only if s is not valid UTF-8.
// within a valid UTF-8 string are encoded as \ufffd replacement characters.
func writeQuotedString(buf *bytes.Buffer, s string) error { func writeQuotedString(buf *bytes.Buffer, s string) error {
if !utf8.ValidString(s) { if !utf8.ValidString(s) {
return fmt.Errorf("interpres: string is not valid UTF-8") return fmt.Errorf("interpres: string is not valid UTF-8")
} }
buf.WriteByte('"') buf.WriteByte('"')
for i := 0; i < len(s); { for i := 0; i < len(s); {
// Write the run of plain characters in one go; the scan stops at a
// byte that needs an escape, a control character, or a multi-byte rune.
j := i
for j < len(s) {
c := s[j]
if c < 0x20 || c == 0x7f || c == '"' || c == '\\' || c >= 0x80 {
break
}
j++
}
if j > i {
buf.WriteString(s[i:j])
i = j
}
if i >= len(s) {
break
}
r, size := utf8.DecodeRuneInString(s[i:]) r, size := utf8.DecodeRuneInString(s[i:])
if r == utf8.RuneError && size == 1 { if r == utf8.RuneError && size == 1 {
buf.WriteString(`\ufffd`) buf.WriteString("\\ufffd")
i++ i++
continue continue
} }
@@ -963,7 +1141,7 @@ func writeEscapedRune(buf *bytes.Buffer, r rune) {
buf.WriteString(`\r`) buf.WriteString(`\r`)
default: default:
if r < 0x20 || r == 0x7f { if r < 0x20 || r == 0x7f {
fmt.Fprintf(buf, `\u%04X`, r) fmt.Fprintf(buf, "\\u%04X", r)
} else { } else {
buf.WriteRune(r) buf.WriteRune(r)
} }
@@ -1034,12 +1212,23 @@ func (e *encoder) writeValue(val any) error {
// writeInlineMap renders m as a TOML inline table, on one line when it fits // writeInlineMap renders m as a TOML inline table, on one line when it fits
// there and across lines when it does not. // there and across lines when it does not.
func (e *encoder) writeInlineMap(m map[string]any) error { func (e *encoder) writeInlineMap(m map[string]any) error {
// An encoder that is itself measuring renders single-line by contract, so
// a nested inline table needs no measuring pass of its own.
if e.limit >= noInlineBreak {
return e.writeInlineMapFlat(m)
}
flat := e.flat() flat := e.flat()
if err := flat.writeInlineMapFlat(m); err != nil { err := flat.writeInlineMapFlat(m)
if err != nil {
flat.release()
return err return err
} }
if e.column()+flat.buf.Len() <= e.limit { fits := e.column()+flat.buf.Len() <= e.limit
if fits {
e.buf.Write(flat.buf.Bytes()) e.buf.Write(flat.buf.Bytes())
}
flat.release()
if fits {
return nil return nil
} }
return e.writeInlineMapMultiline(m) return e.writeInlineMapMultiline(m)
@@ -1180,20 +1369,25 @@ func (e *encoder) writeInlineSubTableIfSmall(name string, doc *tomlDoc) (bool, e
} }
flat := e.flat() flat := e.flat()
if err := flat.writeInlineDoc(doc); err != nil { if err := flat.writeInlineDoc(doc); err != nil {
flat.release()
return false, err return false, err
} }
if flat.buf.Len() > e.opts.inlineTablesAt { if flat.buf.Len() > e.opts.inlineTablesAt {
flat.release()
return false, nil return false, nil
} }
if err := e.writeKey(name); err != nil { if err := e.writeKey(name); err != nil {
flat.release()
return false, err return false, err
} }
e.buf.WriteString(" = ") e.buf.WriteString(" = ")
if e.column()+flat.buf.Len() <= e.limit { if e.column()+flat.buf.Len() <= e.limit {
e.buf.Write(flat.buf.Bytes()) e.buf.Write(flat.buf.Bytes())
} else if err := e.writeInlineDocMultiline(doc); err != nil { } else if err := e.writeInlineDocMultiline(doc); err != nil {
flat.release()
return false, err return false, err
} }
flat.release()
e.buf.WriteByte('\n') e.buf.WriteByte('\n')
return true, nil return true, nil
} }
@@ -1201,9 +1395,9 @@ func (e *encoder) writeInlineSubTableIfSmall(name string, doc *tomlDoc) (bool, e
func (e *encoder) writeStringVal(s string) error { func (e *encoder) writeStringVal(s string) error {
if e.opts.literalMultilineAt > 0 && strings.ContainsRune(s, '\n') && if e.opts.literalMultilineAt > 0 && strings.ContainsRune(s, '\n') &&
len(s) >= e.opts.literalMultilineAt && canBeLiteralMultiline(s) { len(s) >= e.opts.literalMultilineAt && canBeLiteralMultiline(s) {
return writeLiteralMultilineString(&e.buf, s) return writeLiteralMultilineString(e.buf, s)
} }
return writeQuotedString(&e.buf, s) return writeQuotedString(e.buf, s)
} }
// canBeLiteralMultiline reports whether s can be carried verbatim by the // canBeLiteralMultiline reports whether s can be carried verbatim by the
+7 -1
View File
@@ -24,6 +24,7 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"slices"
) )
// A SyntaxError describes a malformed TOML document, including the 1-based // A SyntaxError describes a malformed TOML document, including the 1-based
@@ -423,7 +424,12 @@ func (e *Encoder) MarshalContext(ctx context.Context, v any) ([]byte, error) {
enc.ctx = ctx enc.ctx = ctx
enc.opts = *e enc.opts = *e
if err := enc.encode(v); err != nil { if err := enc.encode(v); err != nil {
enc.release()
return nil, err return nil, err
} }
return enc.bytes(), nil // The output leaves the pooled buffer as a copy, so the next Marshal
// reuses the buffer without touching what the caller holds.
out := slices.Clone(enc.buf.Bytes())
enc.release()
return out, nil
} }