Compare commits
25
Commits
18f1cd51e9
..
v1.1.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ad6c32d0c6 | ||
|
|
17574a0d15 | ||
|
|
8aa2b1b9c0 | ||
|
|
6a043e2824 | ||
|
|
81033bb27c | ||
|
|
dfd5d240d2 | ||
|
|
78946578d1 | ||
|
|
d365729b37 | ||
|
|
53102d70e6 | ||
|
|
f1a757ec5c | ||
|
|
3c8ac859c0 | ||
|
|
4def1b3e8b | ||
|
|
d5327568fb | ||
|
|
c485aab227 | ||
|
|
30b28fe7fc | ||
|
|
aaea68efc9 | ||
|
|
a8d69d90d5 | ||
|
|
bb238c98c3 | ||
|
|
54c6032a9a | ||
|
|
ec0d7a0023 | ||
|
|
feef4fe9ea | ||
|
|
3c1f65038b | ||
|
|
830f840f44 | ||
|
|
696f117c22 | ||
|
|
d2fc31d260 |
@@ -105,5 +105,6 @@ jobs:
|
|||||||
run: go build -o bin/interpres-decode ./cmd/interpres-decode
|
run: go build -o bin/interpres-decode ./cmd/interpres-decode
|
||||||
|
|
||||||
- name: Compliance suite
|
- name: Compliance suite
|
||||||
# interpres implements TOML 1.0; v2 tests 1.1 by default, so the mode is pinned.
|
# interpres implements TOML 1.0 and 1.1; the mode is pinned so an upstream
|
||||||
run: bin/toml-test test -decoder=bin/interpres-decode -toml=1.0
|
# default change cannot silently move the corpus.
|
||||||
|
run: bin/toml-test test -decoder=bin/interpres-decode -toml=1.1
|
||||||
|
|||||||
+70
-2
@@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
|
-
|
||||||
|
|
||||||
|
## [1.1.0] - 2026-09-18
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- TOML 1.1 support, on by default: date-times and times without seconds
|
||||||
|
(`07:32`, `1979-05-27T07:32`, normalised to full seconds on output), the
|
||||||
|
`\e` and `\xHH` escape sequences, and multi-line inline tables with
|
||||||
|
comments and trailing commas. The compliance suite runs in TOML 1.1 mode:
|
||||||
|
214 valid and 467 invalid cases, zero failures. Every TOML 1.0 document
|
||||||
|
parses exactly as before.
|
||||||
- `interpres-decode -validate [file ...]`: a validate mode beside the
|
- `interpres-decode -validate [file ...]`: a validate mode beside the
|
||||||
toml-test adapter. It parses each named file, or stdin when none are named,
|
toml-test adapter. It parses each named file, or stdin when none are named,
|
||||||
prints one line per invalid document to stderr, and exits 0 when all are
|
prints one line per invalid document to stderr, and exits 0 when all are
|
||||||
@@ -29,12 +41,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
- The compliance suite is [toml-test](https://github.com/toml-lang/toml-test)
|
- The compliance suite is [toml-test](https://github.com/toml-lang/toml-test)
|
||||||
v2.2.0, run in TOML 1.0 mode. The new corpus holds 205 valid and 474 invalid
|
v2.2.0, up from v1.6.0. Its TOML 1.0 corpus holds 205 valid and 474 invalid
|
||||||
cases (v1.6.0 had 185 and 371), and it caught the two documents the parser
|
cases (185 and 371 before), and it caught the two documents the parser
|
||||||
still accepted, fixed below.
|
still accepted, fixed below.
|
||||||
|
- The flattened struct layout the decoder consults is cached per struct type
|
||||||
|
and shared with the encoder, which now resolves duplicate field keys with
|
||||||
|
it. Strict decoding of an array of tables of structs runs about a quarter
|
||||||
|
faster; marshalling structs gained the same layout without measurable cost.
|
||||||
|
- The parser scans the input bytes in place instead of building a `[]rune`
|
||||||
|
copy of the document: every character that drives the grammar is ASCII and
|
||||||
|
the input is validated UTF-8 up front, so the conversion pass and its four
|
||||||
|
bytes per rune were pure overhead. Parsing a large array-of-tables document
|
||||||
|
runs about a fifth faster and allocates about half the memory.
|
||||||
|
- Numeric tokens without underscores skip the normalising rebuild: digits are
|
||||||
|
validated in place in `joinDigits`, and a float whose token is already
|
||||||
|
clean goes to `strconv.ParseFloat` directly. One allocation per integer
|
||||||
|
atom and two per float atom disappear.
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
|
- A `MarshalTOML` result of `nil` with a nil error fails the marshal with
|
||||||
|
`MarshalTOML returned a nil value`. The field silently vanished before, and
|
||||||
|
inside a value array the nil result reached reflection as a zero value and
|
||||||
|
panicked.
|
||||||
|
- Strict decoding reports the smallest unknown key. Several unknown keys in
|
||||||
|
one table made the message depend on Go's random map iteration order, so
|
||||||
|
the same document reported different keys across runs.
|
||||||
|
- Decoding into a struct that embeds a pointer to itself terminates. The
|
||||||
|
schema walk recursed through the embedded type forever, so such a
|
||||||
|
`Unmarshal` call hung the process; the walk now tracks the struct types on
|
||||||
|
the current path and stops when one repeats.
|
||||||
|
- An array-of-tables header whose path runs through an inline table
|
||||||
|
(`a = {b = {}}` followed by `[[a.b.c]]`) is rejected. The frozen-inline-table
|
||||||
|
check covered `[table]` headers and dotted keys but not the intermediate
|
||||||
|
steps of an array-of-tables header, so such a document silently extended the
|
||||||
|
inline table.
|
||||||
|
- A new element of an array of tables starts a fresh scope for dotted-key paths
|
||||||
|
and nested arrays of tables: `[[a]]`, `b.c = 1`, `[[a]]`, `[a.b]` parses, as
|
||||||
|
the TOML examples in the spec shape it. The records of the previous element
|
||||||
|
falsely rejected the same paths in the next one.
|
||||||
|
- `Marshal` emits exactly one key when two struct fields resolve to the same
|
||||||
|
TOML name, picking the field the decoder would fill (the shallower one, the
|
||||||
|
later declaration at equal depth). Such a struct previously marshalled into
|
||||||
|
a duplicate key, and the output never re-parsed, breaking the round-trip
|
||||||
|
guarantee.
|
||||||
|
- `Marshal` returns an error for a table header key or an inline-table key that
|
||||||
|
is not valid UTF-8, the way scalar keys already did, instead of silently
|
||||||
|
emitting corrupt TOML (a header that lost its key, an inline table with a
|
||||||
|
missing key).
|
||||||
|
- `UseLiteralMultiline` falls back to the escaped basic string when the value
|
||||||
|
cannot be carried verbatim by the literal form: a run of three single quotes,
|
||||||
|
a control character, or a lone carriage return. Such values previously
|
||||||
|
produced output that did not re-parse.
|
||||||
|
- A `[]any` holding only tables marshals in the value-array form with inline
|
||||||
|
tables, keeping the type `Parse` produces for such an array. It previously
|
||||||
|
took the `[[header]]` form, so a round-trip changed the value's type from
|
||||||
|
`[]any` to `[]map[string]any`.
|
||||||
|
- Decoding into a `uint` destination checks the type's platform width instead
|
||||||
|
of only the fixed widths, so a 32-bit `uint` no longer truncates silently;
|
||||||
|
decoding a finite float beyond the `float32` range is an overflow error
|
||||||
|
instead of a silent infinity.
|
||||||
|
- Struct fields that resolve to one key at equal depth decode through the
|
||||||
|
field declared later, matching the documented rule; the first one won before.
|
||||||
- A float with an exponent marker but no digits (`1e`, `0.0E`) is rejected;
|
- A float with an exponent marker but no digits (`1e`, `0.0E`) is rejected;
|
||||||
the exponent requires at least one digit.
|
the exponent requires at least one digit.
|
||||||
- A date-time offset outside 00:00 through 23:59 is rejected; such offsets
|
- A date-time offset outside 00:00 through 23:59 is rejected; such offsets
|
||||||
|
|||||||
+6
-4
@@ -53,9 +53,11 @@ just test
|
|||||||
7. Open a pull request against `development`.
|
7. Open a pull request against `development`.
|
||||||
|
|
||||||
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The
|
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The
|
||||||
release workflow runs the full gate set including the race detector and
|
release workflow validates the tag, runs the static gates and the test suite
|
||||||
publishes the Gitea release with the matching `CHANGELOG.md` section as its
|
with the coverage floor, and publishes the Gitea release with the matching
|
||||||
notes.
|
`CHANGELOG.md` section as its notes. The race detector is not in that set: race
|
||||||
|
never runs on a push path, and the local `just gates` raced the tree before the
|
||||||
|
tag was cut.
|
||||||
|
|
||||||
## Code style
|
## Code style
|
||||||
|
|
||||||
@@ -114,7 +116,7 @@ Workflows live in `.gitea/workflows/` and run on the project's own runners:
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Test | push or pull request to `development` | format check, vet, modernisation, build, the test suite with the coverage floor, the toml-test compliance suite |
|
| Test | push or pull request to `development` | format check, vet, modernisation, build, the test suite with the coverage floor, the toml-test compliance suite |
|
||||||
| Race | `workflow_dispatch`, by hand | the suite under the race detector, the same race gate the local `just gates` runs |
|
| Race | `workflow_dispatch`, by hand | the suite under the race detector, the same race gate the local `just gates` runs |
|
||||||
| Release | a `v*` tag | the same gates plus the race detector, then the Gitea release created from the `CHANGELOG.md` section |
|
| Release | a `v*` tag | tag validation, format, vet, modernisation, build and the test suite with the coverage floor, then the Gitea release created from the `CHANGELOG.md` section; no race detector |
|
||||||
|
|
||||||
The local equivalent is `just gates`, which is the same set plus the race
|
The local equivalent is `just gates`, which is the same set plus the race
|
||||||
detector.
|
detector.
|
||||||
|
|||||||
@@ -1,17 +1,18 @@
|
|||||||
# interpres
|
# interpres
|
||||||
|
|
||||||
A TOML 1.0 parser and encoder for Go, written with the standard library alone.
|
A TOML 1.0 and 1.1 parser and encoder for Go, written with the standard
|
||||||
`interpres` (Latin for *interpreter*) gives zero-dependency programs an
|
library alone. `interpres` (Latin for *interpreter*) gives zero-dependency
|
||||||
`encoding/json`-style API for reading and writing TOML, and passes the entire
|
programs an `encoding/json`-style API for reading and writing TOML, and passes
|
||||||
official [toml-test](https://github.com/toml-lang/toml-test) suite: 205 valid
|
the entire official [toml-test](https://github.com/toml-lang/toml-test) suite:
|
||||||
and 474 invalid cases, zero failures.
|
214 valid and 467 invalid cases, zero failures.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Full TOML 1.0**: bare, quoted and dotted keys; tables and arrays of tables;
|
- **Full TOML 1.0 and 1.1**: bare, quoted and dotted keys; tables and arrays of
|
||||||
basic and literal strings including multiline; integers in the four radixes
|
tables; basic and literal strings including multiline, with the 1.1 `\e` and
|
||||||
with `_` separators; floats with exponents, `inf` and `nan`; booleans; the
|
`\xHH` escapes; integers in the four radixes with `_` separators; floats with
|
||||||
four date-time kinds; arrays and inline tables.
|
exponents, `inf` and `nan`; booleans; the four date-time kinds, seconds
|
||||||
|
optional as of 1.1; arrays and inline tables, multi-line as of 1.1.
|
||||||
- **Decoding and encoding**: `Parse` for an untyped tree, `Unmarshal` and
|
- **Decoding and encoding**: `Parse` for an untyped tree, `Unmarshal` and
|
||||||
`Marshal` for structs and maps, mirroring `encoding/json`.
|
`Marshal` for structs and maps, mirroring `encoding/json`.
|
||||||
- **Strict decoding**: `NewDecoder().DisallowUnknownFields()` rejects keys that
|
- **Strict decoding**: `NewDecoder().DisallowUnknownFields()` rejects keys that
|
||||||
|
|||||||
+11
-4
@@ -59,24 +59,31 @@ var (
|
|||||||
"2006-01-02T15:04:05Z07:00",
|
"2006-01-02T15:04:05Z07:00",
|
||||||
"2006-01-02 15:04:05.999999999Z07:00",
|
"2006-01-02 15:04:05.999999999Z07:00",
|
||||||
"2006-01-02 15:04:05Z07:00",
|
"2006-01-02 15:04:05Z07:00",
|
||||||
|
// TOML 1.1 makes the seconds optional.
|
||||||
|
"2006-01-02T15:04Z07:00",
|
||||||
|
"2006-01-02 15:04Z07:00",
|
||||||
}
|
}
|
||||||
localDateTimeLayouts = []string{
|
localDateTimeLayouts = []string{
|
||||||
"2006-01-02T15:04:05.999999999",
|
"2006-01-02T15:04:05.999999999",
|
||||||
"2006-01-02T15:04:05",
|
"2006-01-02T15:04:05",
|
||||||
"2006-01-02 15:04:05.999999999",
|
"2006-01-02 15:04:05.999999999",
|
||||||
"2006-01-02 15:04:05",
|
"2006-01-02 15:04:05",
|
||||||
|
"2006-01-02T15:04",
|
||||||
|
"2006-01-02 15:04",
|
||||||
}
|
}
|
||||||
localTimeLayouts = []string{
|
localTimeLayouts = []string{
|
||||||
"15:04:05.999999999",
|
"15:04:05.999999999",
|
||||||
"15:04:05",
|
"15:04:05",
|
||||||
|
"15:04",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
// dateTimeShape enforces the strict TOML grammar (two-digit components) that
|
// dateTimeShape enforces the strict TOML grammar (two-digit components,
|
||||||
// time.Parse would otherwise accept loosely (e.g. a single-digit hour).
|
// seconds optional since 1.1, a fraction only after seconds) that time.Parse
|
||||||
|
// would otherwise accept loosely (e.g. a single-digit hour).
|
||||||
var dateTimeShape = regexp.MustCompile(
|
var dateTimeShape = regexp.MustCompile(
|
||||||
`^\d{4}-\d{2}-\d{2}([Tt ]\d{2}:\d{2}:\d{2}(\.\d+)?([Zz]|[+-]\d{2}:\d{2})?)?$` +
|
`^\d{4}-\d{2}-\d{2}([Tt ]\d{2}:\d{2}(:\d{2}(\.\d+)?)?([Zz]|[+-]\d{2}:\d{2})?)?$` +
|
||||||
`|^\d{2}:\d{2}:\d{2}(\.\d+)?$`,
|
`|^\d{2}:\d{2}(:\d{2}(\.\d+)?)?$`,
|
||||||
)
|
)
|
||||||
|
|
||||||
// offsetBounds extracts the numeric offset of a date-time. The ABNF bounds it
|
// offsetBounds extracts the numeric offset of a date-time. The ABNF bounds it
|
||||||
|
|||||||
@@ -5,9 +5,10 @@ package interpres
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"math"
|
|
||||||
"reflect"
|
"reflect"
|
||||||
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -105,13 +106,26 @@ func (d *decoder) assignTable(tbl map[string]any, dst reflect.Value) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (d *decoder) assignStruct(tbl map[string]any, dst reflect.Value) error {
|
func (d *decoder) assignStruct(tbl map[string]any, dst reflect.Value) error {
|
||||||
schema := newStructSchema(dst.Type())
|
schema := cachedStructSchema(dst.Type())
|
||||||
|
if d.disallowUnknown {
|
||||||
|
// Map iteration order is random, so pick the unknown key to report
|
||||||
|
// deterministically: the smallest one.
|
||||||
|
unknown := ""
|
||||||
|
for key := range tbl {
|
||||||
|
if _, ok := schema.byName[strings.ToLower(key)]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if unknown == "" || key < unknown {
|
||||||
|
unknown = key
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if unknown != "" {
|
||||||
|
return fmt.Errorf("interpres: unknown field %q for %s", unknown, dst.Type())
|
||||||
|
}
|
||||||
|
}
|
||||||
for key, val := range tbl {
|
for key, val := range tbl {
|
||||||
field, ok := schema.byName[strings.ToLower(key)]
|
field, ok := schema.byName[strings.ToLower(key)]
|
||||||
if !ok {
|
if !ok {
|
||||||
if d.disallowUnknown {
|
|
||||||
return fmt.Errorf("interpres: unknown field %q for %s", key, dst.Type())
|
|
||||||
}
|
|
||||||
if schema.embedMaps != nil {
|
if schema.embedMaps != nil {
|
||||||
// Leftover keys land in an untagged embedded map, the inverse
|
// Leftover keys land in an untagged embedded map, the inverse
|
||||||
// of the encoder inlining that map's entries.
|
// of the encoder inlining that map's entries.
|
||||||
@@ -203,21 +217,21 @@ func setInt(dst reflect.Value, v int64) error {
|
|||||||
if v < 0 {
|
if v < 0 {
|
||||||
return fmt.Errorf("interpres: cannot assign negative %d to %s", v, dst.Type())
|
return fmt.Errorf("interpres: cannot assign negative %d to %s", v, dst.Type())
|
||||||
}
|
}
|
||||||
var max uint64
|
// OverflowUint knows every width, uint included on platforms where it
|
||||||
switch dst.Kind() {
|
// is narrower than uint64; SetUint would silently truncate instead.
|
||||||
case reflect.Uint8:
|
if dst.OverflowUint(uint64(v)) {
|
||||||
max = math.MaxUint8
|
|
||||||
case reflect.Uint16:
|
|
||||||
max = math.MaxUint16
|
|
||||||
case reflect.Uint32:
|
|
||||||
max = math.MaxUint32
|
|
||||||
}
|
|
||||||
if max != 0 && uint64(v) > max {
|
|
||||||
return fmt.Errorf("interpres: integer %d overflows %s", v, dst.Type())
|
return fmt.Errorf("interpres: integer %d overflows %s", v, dst.Type())
|
||||||
}
|
}
|
||||||
dst.SetUint(uint64(v))
|
dst.SetUint(uint64(v))
|
||||||
case reflect.Float32, reflect.Float64:
|
case reflect.Float32, reflect.Float64:
|
||||||
dst.SetFloat(float64(v))
|
// A finite value beyond the float32 range would silently become ±Inf;
|
||||||
|
// infinities and NaN themselves pass through. An int64 never
|
||||||
|
// overflows either float width.
|
||||||
|
f := float64(v)
|
||||||
|
if dst.OverflowFloat(f) {
|
||||||
|
return fmt.Errorf("interpres: integer %d overflows %s", v, dst.Type())
|
||||||
|
}
|
||||||
|
dst.SetFloat(f)
|
||||||
default:
|
default:
|
||||||
return fmt.Errorf("interpres: cannot assign integer to %s", dst.Type())
|
return fmt.Errorf("interpres: cannot assign integer to %s", dst.Type())
|
||||||
}
|
}
|
||||||
@@ -227,6 +241,9 @@ func setInt(dst reflect.Value, v int64) error {
|
|||||||
func setFloat(dst reflect.Value, v float64) error {
|
func setFloat(dst reflect.Value, v float64) error {
|
||||||
switch dst.Kind() {
|
switch dst.Kind() {
|
||||||
case reflect.Float32, reflect.Float64:
|
case reflect.Float32, reflect.Float64:
|
||||||
|
if dst.OverflowFloat(v) {
|
||||||
|
return fmt.Errorf("interpres: float %g overflows %s", v, dst.Type())
|
||||||
|
}
|
||||||
dst.SetFloat(v)
|
dst.SetFloat(v)
|
||||||
return nil
|
return nil
|
||||||
default:
|
default:
|
||||||
@@ -253,10 +270,32 @@ type structSchema struct {
|
|||||||
embedMaps [][]int
|
embedMaps [][]int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// structSchemaCache holds one schema per struct type. A schema is immutable
|
||||||
|
// once published, so concurrent callers only race to build an identical value,
|
||||||
|
// the same trade-off encoding/json's field cache makes. The cache grows with
|
||||||
|
// the number of distinct types decoded or encoded, never per document.
|
||||||
|
var structSchemaCache sync.Map // reflect.Type -> structSchema
|
||||||
|
|
||||||
|
func cachedStructSchema(t reflect.Type) structSchema {
|
||||||
|
if s, ok := structSchemaCache.Load(t); ok {
|
||||||
|
return s.(structSchema)
|
||||||
|
}
|
||||||
|
s := newStructSchema(t)
|
||||||
|
actual, _ := structSchemaCache.LoadOrStore(t, s)
|
||||||
|
return actual.(structSchema)
|
||||||
|
}
|
||||||
|
|
||||||
func newStructSchema(t reflect.Type) structSchema {
|
func newStructSchema(t reflect.Type) structSchema {
|
||||||
s := structSchema{byName: make(map[string]structFieldLoc, t.NumField())}
|
s := structSchema{byName: make(map[string]structFieldLoc, t.NumField())}
|
||||||
|
// A struct may embed a pointer to itself, which is legal Go, so the walk
|
||||||
|
// tracks the struct types on the current path and stops when one repeats;
|
||||||
|
// without the guard the recursion never terminates. A self-promoted key
|
||||||
|
// always loses to the shallower original, so skipping it changes nothing.
|
||||||
|
visiting := map[reflect.Type]bool{}
|
||||||
var walk func(t reflect.Type, prefix []int, depth int)
|
var walk func(t reflect.Type, prefix []int, depth int)
|
||||||
walk = func(t reflect.Type, prefix []int, depth int) {
|
walk = func(t reflect.Type, prefix []int, depth int) {
|
||||||
|
visiting[t] = true
|
||||||
|
defer delete(visiting, t)
|
||||||
for i := range t.NumField() {
|
for i := range t.NumField() {
|
||||||
f := t.Field(i)
|
f := t.Field(i)
|
||||||
if f.PkgPath != "" { // unexported
|
if f.PkgPath != "" { // unexported
|
||||||
@@ -277,7 +316,9 @@ func newStructSchema(t reflect.Type) structSchema {
|
|||||||
}
|
}
|
||||||
switch {
|
switch {
|
||||||
case ft.Kind() == reflect.Struct && !isScalarStruct(ft):
|
case ft.Kind() == reflect.Struct && !isScalarStruct(ft):
|
||||||
walk(ft, path, depth+1)
|
if !visiting[ft] {
|
||||||
|
walk(ft, path, depth+1)
|
||||||
|
}
|
||||||
continue
|
continue
|
||||||
case ft.Kind() == reflect.Map && ft.Key().Kind() == reflect.String:
|
case ft.Kind() == reflect.Map && ft.Key().Kind() == reflect.String:
|
||||||
s.embedMaps = append(s.embedMaps, path)
|
s.embedMaps = append(s.embedMaps, path)
|
||||||
@@ -289,7 +330,7 @@ func newStructSchema(t reflect.Type) structSchema {
|
|||||||
name = f.Name
|
name = f.Name
|
||||||
}
|
}
|
||||||
key := strings.ToLower(name)
|
key := strings.ToLower(name)
|
||||||
if existing, ok := s.byName[key]; !ok || depth < existing.depth {
|
if existing, ok := s.byName[key]; !ok || depth <= existing.depth {
|
||||||
s.byName[key] = structFieldLoc{index: path, depth: depth}
|
s.byName[key] = structFieldLoc{index: path, depth: depth}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -298,6 +339,15 @@ func newStructSchema(t reflect.Type) structSchema {
|
|||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ownsKey reports whether the field at path is the one that resolves key.
|
||||||
|
// The encoder consults it to emit exactly the field the decoder would fill,
|
||||||
|
// so a struct with two fields mapping to one key does not marshal into a
|
||||||
|
// duplicate TOML key.
|
||||||
|
func (s structSchema) ownsKey(key string, path []int) bool {
|
||||||
|
loc, ok := s.byName[key]
|
||||||
|
return ok && slices.Equal(loc.index, path)
|
||||||
|
}
|
||||||
|
|
||||||
// fieldByIndex walks an index path from a struct value, allocating nil
|
// fieldByIndex walks an index path from a struct value, allocating nil
|
||||||
// pointers along the way so a key can reach through an embedded pointer
|
// pointers along the way so a key can reach through an embedded pointer
|
||||||
// struct. Every field on the path is exported, so each step is settable.
|
// struct. Every field on the path is exported, so each step is settable.
|
||||||
|
|||||||
@@ -223,6 +223,33 @@ func TestUnmarshalIntToUint64FitsMaxInt64(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUnmarshalFloat32Overflow(t *testing.T) {
|
||||||
|
// A finite float64 beyond the float32 range must not decode silently as
|
||||||
|
// an infinity.
|
||||||
|
type C struct {
|
||||||
|
X float32 `toml:"x"`
|
||||||
|
}
|
||||||
|
var c C
|
||||||
|
err := Unmarshal([]byte("x = 1e300\n"), &c)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected overflow error for float32")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "overflow") {
|
||||||
|
t.Errorf("err = %v, want substring 'overflow'", err.Error())
|
||||||
|
}
|
||||||
|
// Infinities themselves pass through, and in-range values are untouched.
|
||||||
|
var ok C
|
||||||
|
if err := Unmarshal([]byte("x = inf\n"), &ok); err != nil {
|
||||||
|
t.Fatalf("inf should decode into float32, got %v", err)
|
||||||
|
}
|
||||||
|
if !math.IsInf(float64(ok.X), 1) {
|
||||||
|
t.Errorf("X = %v, want +Inf", ok.X)
|
||||||
|
}
|
||||||
|
if err := Unmarshal([]byte("x = 1.5\n"), &ok); err != nil || ok.X != 1.5 {
|
||||||
|
t.Fatalf("1.5 should decode into float32, got %v (X=%v)", err, ok.X)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestUnmarshalNegativeIntToUint(t *testing.T) {
|
func TestUnmarshalNegativeIntToUint(t *testing.T) {
|
||||||
type C struct {
|
type C struct {
|
||||||
X uint8 `toml:"x"`
|
X uint8 `toml:"x"`
|
||||||
@@ -538,6 +565,34 @@ func TestUnmarshalEmbeddedPointerStruct(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A struct embedding a pointer to itself is legal Go; decoding into it must
|
||||||
|
// terminate. The schema walk used to recurse through the embedded type
|
||||||
|
// forever.
|
||||||
|
func TestUnmarshalSelfEmbeddedPointerStructTerminates(t *testing.T) {
|
||||||
|
type SelfLink struct {
|
||||||
|
*SelfLink
|
||||||
|
X int `toml:"x"`
|
||||||
|
Y string `toml:"y"`
|
||||||
|
}
|
||||||
|
var n SelfLink
|
||||||
|
if err := Unmarshal([]byte("x = 1\ny = \"s\"\n"), &n); err != nil {
|
||||||
|
t.Fatalf("unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if n.X != 1 || n.Y != "s" {
|
||||||
|
t.Fatalf("decoded: %+v", n)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A nil self pointer on the encode side stays skippable, as any nil
|
||||||
|
// embedded pointer is.
|
||||||
|
out, err := Marshal(SelfLink{X: 2})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal: %v", err)
|
||||||
|
}
|
||||||
|
if want := "x = 2\ny = \"\"\n"; string(out) != want {
|
||||||
|
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
type RoundTripExtra map[string]int
|
type RoundTripExtra map[string]int
|
||||||
|
|
||||||
type RoundTripMapCfg struct {
|
type RoundTripMapCfg struct {
|
||||||
@@ -586,6 +641,22 @@ func TestUnmarshalEmbeddedNameClashShallowerWins(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUnmarshalNameClashEqualDepthLaterWins(t *testing.T) {
|
||||||
|
// At equal depth the field declared later resolves the name, matching the
|
||||||
|
// documented rule.
|
||||||
|
type C struct {
|
||||||
|
First string `toml:"v"`
|
||||||
|
Second int `toml:"v"`
|
||||||
|
}
|
||||||
|
var c C
|
||||||
|
if err := Unmarshal([]byte("v = 1\n"), &c); err != nil {
|
||||||
|
t.Fatalf("unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if c.Second != 1 {
|
||||||
|
t.Fatalf("decoded: %+v, want the later field to take the value", c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestUnmarshalUnknownKeyWithoutEmbeddedMap(t *testing.T) {
|
func TestUnmarshalUnknownKeyWithoutEmbeddedMap(t *testing.T) {
|
||||||
var cfg RoundTripDerived
|
var cfg RoundTripDerived
|
||||||
if err := Unmarshal([]byte("rogue = 1\n"), &cfg); err != nil {
|
if err := Unmarshal([]byte("rogue = 1\n"), &cfg); err != nil {
|
||||||
|
|||||||
+27
-8
@@ -7,6 +7,11 @@ package. The snippets assume:
|
|||||||
import "sourcedock.dev/petrbalvin/interpres"
|
import "sourcedock.dev/petrbalvin/interpres"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The parser accepts TOML 1.0 documents plus the TOML 1.1 extensions: date-times
|
||||||
|
and times without seconds, the `\e` and `\xHH` escape sequences, and
|
||||||
|
multi-line inline tables with comments and trailing commas. The encoder emits
|
||||||
|
TOML 1.0, which is valid under both versions.
|
||||||
|
|
||||||
## Functions
|
## Functions
|
||||||
|
|
||||||
### `func Parse(data []byte) (map[string]any, error)`
|
### `func Parse(data []byte) (map[string]any, error)`
|
||||||
@@ -46,7 +51,7 @@ The cancellable variant of `Unmarshal`.
|
|||||||
### `func Marshal(v any) ([]byte, error)`
|
### `func Marshal(v any) ([]byte, error)`
|
||||||
|
|
||||||
Encodes a `struct` or `map[string]V` value, or a non-nil pointer to one, into a
|
Encodes a `struct` or `map[string]V` value, or a non-nil pointer to one, into a
|
||||||
TOML 1.0 document. The emission rules are in the [Encoding](#encoding) section
|
TOML document. The emission rules are in the [Encoding](#encoding) section
|
||||||
below. Equivalent to `MarshalContext(context.Background(), v)`.
|
below. Equivalent to `MarshalContext(context.Background(), v)`.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
@@ -126,8 +131,8 @@ The decoder converts to the destination type with explicit overflow checks:
|
|||||||
| Destination kind | Rule |
|
| Destination kind | Rule |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `int`, `int8`, `int16`, `int32`, `int64` | the `int64` value must not overflow the destination |
|
| `int`, `int8`, `int16`, `int32`, `int64` | the `int64` value must not overflow the destination |
|
||||||
| `uint`, `uint8`, `uint16`, `uint32`, `uint64` | the value must be non-negative; `uint8`, `uint16` and `uint32` enforce their own maxima; `uint64` accepts any non-negative `int64` |
|
| `uint`, `uint8`, `uint16`, `uint32`, `uint64` | the value must be non-negative and must not overflow the destination's own width, `uint` on a 32-bit platform included; `uint64` accepts any non-negative `int64` |
|
||||||
| `float32`, `float64` | copied verbatim; an integer also coerces, so TOML `5` decodes into `5.0` |
|
| `float32`, `float64` | copied verbatim, except that a finite value beyond the `float32` range is an overflow error rather than a silent infinity; an integer also coerces, so TOML `5` decodes into `5.0` |
|
||||||
| `bool`, `string` | exact kind match only, no coercion across kinds |
|
| `bool`, `string` | exact kind match only, no coercion across kinds |
|
||||||
| `time.Time` | offset date-times only; no implicit conversion to or from the local variants |
|
| `time.Time` | offset date-times only; no implicit conversion to or from the local variants |
|
||||||
|
|
||||||
@@ -139,8 +144,10 @@ offending key or index, for example `p: interpres: integer 300 overflows uint8`.
|
|||||||
Offset date-times decode into `time.Time` and keep their offset. The local
|
Offset date-times decode into `time.Time` and keep their offset. The local
|
||||||
variants decode into `LocalDateTime`, `LocalDate` and `LocalTime`, whose
|
variants decode into `LocalDateTime`, `LocalDate` and `LocalTime`, whose
|
||||||
embedded `time.Time` is normalised to UTC (midnight UTC for a local date, the
|
embedded `time.Time` is normalised to UTC (midnight UTC for a local date, the
|
||||||
zero date for a local time). There is no implicit conversion between the offset
|
zero date for a local time). Every kind may omit the seconds as of TOML 1.1
|
||||||
and local kinds; assigning one to the other is an error.
|
(`07:32`, `1979-05-27T07:32`); such a value carries a zero second, and the
|
||||||
|
canonical rendering writes full seconds. There is no implicit conversion
|
||||||
|
between the offset and local kinds; assigning one to the other is an error.
|
||||||
|
|
||||||
### Arrays of tables
|
### Arrays of tables
|
||||||
|
|
||||||
@@ -186,7 +193,8 @@ A typo such as `database_urls` then fails with
|
|||||||
`interpres: unknown field "database_urls" for main.Config` instead of a silent
|
`interpres: unknown field "database_urls" for main.Config` instead of a silent
|
||||||
default-zero run. Strictness applies to every struct the decode reaches, at any
|
default-zero run. Strictness applies to every struct the decode reaches, at any
|
||||||
depth, including struct elements inside slices; map destinations accept every
|
depth, including struct elements inside slices; map destinations accept every
|
||||||
key by nature.
|
key by nature. When several keys are unknown, the message names the smallest
|
||||||
|
one, so it does not depend on map iteration order.
|
||||||
|
|
||||||
### Cancellation
|
### Cancellation
|
||||||
|
|
||||||
@@ -309,7 +317,10 @@ The returned value is encoded as if it had been passed in place of the
|
|||||||
receiver, so it may be a scalar, a slice, an array of tables, or another
|
receiver, so it may be a scalar, a slice, an array of tables, or another
|
||||||
struct or map, including the `Marshaler` result of another type; the encoder
|
struct or map, including the `Marshaler` result of another type; the encoder
|
||||||
recurses. An error returned from `MarshalTOML` fails the marshal wrapped with
|
recurses. An error returned from `MarshalTOML` fails the marshal wrapped with
|
||||||
the key path, for example `interpres: server.port: bad timestamp`.
|
the key path, for example `interpres: server.port: bad timestamp`. A result
|
||||||
|
of `nil` with a nil error fails the same way with
|
||||||
|
`MarshalTOML returned a nil value`: nil has no TOML representation, so
|
||||||
|
dropping the field silently is not an option.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
type Port int
|
type Port int
|
||||||
@@ -331,6 +342,11 @@ tree, _ := interpres.Parse([]byte(`arr = [1, {a = 2}, "x"]`))
|
|||||||
out, _ := interpres.Marshal(tree) // arr = [1, {a = 2}, "x"]
|
out, _ := interpres.Marshal(tree) // arr = [1, {a = 2}, "x"]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
A `[]any` holding only tables keeps the value-array form as well, because that
|
||||||
|
is the shape `Parse` gives a value array of inline tables; emitting it as
|
||||||
|
`[[headers]]` would re-parse as `[]map[string]any` and change the value's type
|
||||||
|
across a round-trip.
|
||||||
|
|
||||||
### Empty arrays
|
### Empty arrays
|
||||||
|
|
||||||
A nil slice is always omitted. An empty (length 0) array of tables is always
|
A nil slice is always omitted. An empty (length 0) array of tables is always
|
||||||
@@ -351,7 +367,10 @@ out, err := interpres.NewEncoder().UseLiteralMultiline(80).Marshal(cfg)
|
|||||||
```
|
```
|
||||||
|
|
||||||
Single-line strings keep the basic form regardless of the threshold, and a
|
Single-line strings keep the basic form regardless of the threshold, and a
|
||||||
threshold of `0` or less disables the option.
|
threshold of `0` or less disables the option. A string the literal form cannot
|
||||||
|
carry verbatim (an embedded run of three single quotes, a control character
|
||||||
|
other than tab or newline, or a carriage return outside a CRLF pair) also keeps
|
||||||
|
the basic form, so the output always re-parses to the same value.
|
||||||
|
|
||||||
### Cancellation
|
### Cancellation
|
||||||
|
|
||||||
|
|||||||
+13
-7
@@ -6,9 +6,9 @@ source tree; nothing is aspirational.
|
|||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
interpres is one public library package, one command, and one example. The
|
interpres is one public library package, one command, and one example. The
|
||||||
library implements the whole of TOML 1.0, decoding and encoding, in the
|
library implements the whole of TOML 1.0 and 1.1, decoding and encoding, in the
|
||||||
standard library alone; the command wraps the parser for the toml-test
|
standard library alone; the command wraps the parser for the toml-test
|
||||||
compliance harness, against which it stands at 185 valid and 371 invalid cases
|
compliance harness, against which it stands at 214 valid and 467 invalid cases
|
||||||
with zero failures; the example demonstrates the API.
|
with zero failures; the example demonstrates the API.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
@@ -43,7 +43,7 @@ Inside the library package, one file owns one concern:
|
|||||||
|
|
||||||
| File | Responsibility |
|
| File | Responsibility |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `parser.go` | The recursive-descent parser. Produces the `map[string]any` tree and enforces the structural rules of TOML 1.0 (table redefinitions, dotted keys, arrays of tables). Reports a 1-based line on failure. |
|
| `parser.go` | The recursive-descent parser. Produces the `map[string]any` tree and enforces the structural rules of TOML 1.0 and 1.1 (table redefinitions, dotted keys, arrays of tables, multi-line inline tables). Reports a 1-based line on failure. |
|
||||||
| `number.go` | Strict numeric tokens: integers in the four radixes with `_` separators, and floats including `inf` and `nan`. Rejects leading zeros, misplaced underscores and malformed fractions. |
|
| `number.go` | Strict numeric tokens: integers in the four radixes with `_` separators, and floats including `inf` and `nan`. Rejects leading zeros, misplaced underscores and malformed fractions. |
|
||||||
| `datetime.go` | The three local date-time wrapper types and `parseDateTime`, which classifies a token into the four date-time kinds under the strict TOML grammar. |
|
| `datetime.go` | The three local date-time wrapper types and `parseDateTime`, which classifies a token into the four date-time kinds under the strict TOML grammar. |
|
||||||
| `decode.go` | Maps the parsed tree onto Go values by reflection: struct fields, maps, slices, scalar conversion with overflow checks, `Unmarshaler` dispatch. |
|
| `decode.go` | Maps the parsed tree onto Go values by reflection: struct fields, maps, slices, scalar conversion with overflow checks, `Unmarshaler` dispatch. |
|
||||||
@@ -99,11 +99,17 @@ sequenceDiagram
|
|||||||
`Decode`, `DecodeContext`, `Marshal` and `MarshalContext` call allocates its
|
`Decode`, `DecodeContext`, `Marshal` and `MarshalContext` call allocates its
|
||||||
own unexported worker, so a configured type is safe for concurrent use; the
|
own unexported worker, so a configured type is safe for concurrent use; the
|
||||||
setter methods are not, and must finish before the value is shared.
|
setter methods are not, and must finish before the value is shared.
|
||||||
- The parser is allocated per `ParseContext` call; nothing is cached between
|
- The parser is allocated per `ParseContext` call; the parser itself caches
|
||||||
documents.
|
nothing between documents.
|
||||||
|
- The one piece of shared state is the struct-schema cache in `decode.go`: a
|
||||||
|
`sync.Map` keyed by `reflect.Type`, holding the flattened field layout the
|
||||||
|
decoder and the encoder both consult. A schema is immutable once published,
|
||||||
|
so concurrent callers only race to build an identical value, the same
|
||||||
|
trade-off `encoding/json`'s field cache makes. The cache grows with the
|
||||||
|
number of distinct struct types, never with document size.
|
||||||
- The date-time wrappers are values, not pointers, and are immutable in use.
|
- The date-time wrappers are values, not pointers, and are immutable in use.
|
||||||
- Nothing in the library starts goroutines or holds locks; concurrency safety
|
- Nothing in the library starts goroutines; apart from the schema cache above,
|
||||||
comes from having no shared mutable state.
|
which never mutates a published entry, there is no shared mutable state.
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# Benchmarking
|
||||||
|
|
||||||
|
How the performance numbers attached to this project are measured, so that a
|
||||||
|
number in a changelog entry or a release note can be reproduced and trusted.
|
||||||
|
|
||||||
|
## The suite
|
||||||
|
|
||||||
|
The benchmarks live in `bench_test.go`, next to the code they measure:
|
||||||
|
|
||||||
|
| Benchmark | What it measures |
|
||||||
|
|---|---|
|
||||||
|
| `BenchmarkParse` | `Parse` over a representative configuration document |
|
||||||
|
| `BenchmarkMarshal` | `Marshal` of the tree `Parse` produced from the same document |
|
||||||
|
| `BenchmarkStrictDecode` | `Decode` into a struct under `DisallowUnknownFields` |
|
||||||
|
| `BenchmarkParseLong` | `Parse` over a generated document with about 2000 array-of-tables entries |
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```sh
|
||||||
|
just bench
|
||||||
|
```
|
||||||
|
|
||||||
|
The recipe runs the suite with `-benchmem -count=5`. Every benchmark uses
|
||||||
|
`b.Loop`, so setup runs outside the timed region, and `ReportAllocs` records
|
||||||
|
allocations per operation. The parse and marshal benchmarks set `SetBytes`, so
|
||||||
|
their results read as input bytes per second.
|
||||||
|
|
||||||
|
## Method
|
||||||
|
|
||||||
|
- An idle machine only: a loaded box times whatever else is running, and the
|
||||||
|
fastest sample can land on the wrong function.
|
||||||
|
- An A/B comparison runs both variants inside one process, in one binary;
|
||||||
|
separate processes of identical binaries differ by more than the effect
|
||||||
|
being measured.
|
||||||
|
- The five counts are compared through their medians, allocations and bytes
|
||||||
|
per operation alongside the times. Differences within 1 to 2 percent are
|
||||||
|
noise; only a difference beyond that is a result.
|
||||||
|
- When timing is hopeless, the allocation and byte counts are the result.
|
||||||
|
|
||||||
|
## Reports
|
||||||
|
|
||||||
|
The repository stores no benchmark reports. A performance claim in
|
||||||
|
`CHANGELOG.md` is measured with the method above on the change that makes it,
|
||||||
|
and the number travels with the claim.
|
||||||
+9
-6
@@ -77,7 +77,9 @@ just bench
|
|||||||
```
|
```
|
||||||
|
|
||||||
Benchmark on an idle machine, and compare only runs made in one process against
|
Benchmark on an idle machine, and compare only runs made in one process against
|
||||||
each other. The recipe sweeps `./...` five times with `-benchmem`.
|
each other. The recipe sweeps `./...` five times with `-benchmem`. The binding
|
||||||
|
measurement method, and what counts as a result, is in
|
||||||
|
[docs/BENCHMARKING.md](BENCHMARKING.md).
|
||||||
|
|
||||||
## Debugging the build
|
## Debugging the build
|
||||||
|
|
||||||
@@ -97,12 +99,13 @@ pipeline.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `test.yml` | push or pull request to `development` | format check, vet, modernisation, build, the test suite with the 80 percent coverage floor, then the toml-test compliance suite |
|
| `test.yml` | push or pull request to `development` | format check, vet, modernisation, build, the test suite with the 80 percent coverage floor, then the toml-test compliance suite |
|
||||||
| `race.yml` | `workflow_dispatch`, by hand | the suite under the race detector; the same race gate `just gates` runs locally |
|
| `race.yml` | `workflow_dispatch`, by hand | the suite under the race detector; the same race gate `just gates` runs locally |
|
||||||
| `release.yml` | a `v*` tag | the same gates plus the race detector, then the Gitea release from the CHANGELOG section |
|
| `release.yml` | a `v*` tag | tag validation, then format, vet, modernisation, build and the test suite with the coverage floor, then the Gitea release from the CHANGELOG section. No race detector: race never runs on a push path, and the local `just gates` raced the tree before the tag was cut |
|
||||||
|
|
||||||
## Releases
|
## Releases
|
||||||
|
|
||||||
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The
|
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The
|
||||||
tag drives the release workflow: it validates the tag, runs the full gate set
|
tag drives the release workflow: it validates the tag, runs the static gates
|
||||||
including the race detector, extracts the matching `## [X.Y.Z]` section from
|
and the test suite with the coverage floor, extracts the matching `## [X.Y.Z]`
|
||||||
`CHANGELOG.md`, and publishes the release with that section as its body. A
|
section from `CHANGELOG.md`, and publishes the release with that section as its
|
||||||
library ships no binaries, so the release carries the notes and nothing else.
|
body. A library ships no binaries, so the release carries the notes and nothing
|
||||||
|
else.
|
||||||
|
|||||||
@@ -144,6 +144,16 @@ func (d *tomlDoc) partitionedEntries() (scalars []entry, tables []entry, arrays
|
|||||||
// --- reflection walk: struct ---------------------------------------------
|
// --- reflection walk: struct ---------------------------------------------
|
||||||
|
|
||||||
func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
||||||
|
return walkStructDoc(v, doc, ctx, nil, cachedStructSchema(v.Type()))
|
||||||
|
}
|
||||||
|
|
||||||
|
// walkStructDoc emits the fields of v into doc. prefix is v's index path from
|
||||||
|
// the struct whose schema resolves key conflicts; an embedded struct is walked
|
||||||
|
// with the outer schema and a longer prefix, so every leaf competes under the
|
||||||
|
// decoder's rule: the shallower field wins, the later declaration at equal
|
||||||
|
// depth. A field another field shadows is skipped, because emitting both
|
||||||
|
// would duplicate the key and the output would not re-parse.
|
||||||
|
func walkStructDoc(v reflect.Value, doc *tomlDoc, ctx string, prefix []int, schema structSchema) error {
|
||||||
t := v.Type()
|
t := v.Type()
|
||||||
for i := range t.NumField() {
|
for i := range t.NumField() {
|
||||||
if i%ctxCheckInterval == 0 {
|
if i%ctxCheckInterval == 0 {
|
||||||
@@ -155,6 +165,7 @@ func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
|||||||
if f.PkgPath != "" {
|
if f.PkgPath != "" {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
path := append(append([]int{}, prefix...), i)
|
||||||
if f.Anonymous {
|
if f.Anonymous {
|
||||||
tag, _ := f.Tag.Lookup("toml")
|
tag, _ := f.Tag.Lookup("toml")
|
||||||
if tag == "-" {
|
if tag == "-" {
|
||||||
@@ -169,12 +180,15 @@ func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
|||||||
case reflect.Struct:
|
case reflect.Struct:
|
||||||
if isScalarStruct(fv.Type()) {
|
if isScalarStruct(fv.Type()) {
|
||||||
name := strings.ToLower(f.Name)
|
name := strings.ToLower(f.Name)
|
||||||
|
if !schema.ownsKey(name, path) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
if err := doc.appendScalar(name, fv.Interface(), ctx); err != nil {
|
if err := doc.appendScalar(name, fv.Interface(), ctx); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if err := buildStructDoc(fv, doc, ctx); err != nil {
|
if err := walkStructDoc(fv, doc, ctx, path, schema); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
@@ -190,6 +204,9 @@ func buildStructDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
|||||||
if name == "-" {
|
if name == "-" {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
if !schema.ownsKey(strings.ToLower(name), path) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
if fieldOmitted(f, v.Field(i)) {
|
if fieldOmitted(f, v.Field(i)) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -283,6 +300,12 @@ func buildMapDoc(v reflect.Value, doc *tomlDoc, ctx string) error {
|
|||||||
|
|
||||||
// --- reflection walk: field dispatch -------------------------------------
|
// --- reflection walk: field dispatch -------------------------------------
|
||||||
|
|
||||||
|
// errNilMarshalTOML reports a Marshaler whose method returned a nil value
|
||||||
|
// with no error. nil has no TOML representation, so dropping the field
|
||||||
|
// silently or panicking on the invalid reflect.Value would both hide the
|
||||||
|
// contract violation.
|
||||||
|
var errNilMarshalTOML = errors.New("MarshalTOML returned a nil value")
|
||||||
|
|
||||||
func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
|
func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
|
||||||
if v.CanInterface() {
|
if v.CanInterface() {
|
||||||
if m, ok := v.Interface().(Marshaler); ok {
|
if m, ok := v.Interface().(Marshaler); ok {
|
||||||
@@ -290,6 +313,9 @@ func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return &EncodeError{Path: joinKey(ctx, name), Err: err}
|
return &EncodeError{Path: joinKey(ctx, name), Err: err}
|
||||||
}
|
}
|
||||||
|
if mv == nil {
|
||||||
|
return &EncodeError{Path: joinKey(ctx, name), Err: errNilMarshalTOML}
|
||||||
|
}
|
||||||
v = reflect.ValueOf(mv)
|
v = reflect.ValueOf(mv)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -371,6 +397,13 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
|
|||||||
break
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// A []any of tables is what Parse produces for a value array of inline
|
||||||
|
// tables; the [[header]] form would re-parse as []map[string]any and so
|
||||||
|
// change the value's Go type across a round-trip. The header form is
|
||||||
|
// reserved for typed table slices.
|
||||||
|
if v.Type().Elem().Kind() == reflect.Interface {
|
||||||
|
allTables = false
|
||||||
|
}
|
||||||
if allTables {
|
if allTables {
|
||||||
subs := make([]*tomlDoc, n)
|
subs := make([]*tomlDoc, n)
|
||||||
for i := range n {
|
for i := range n {
|
||||||
@@ -424,6 +457,9 @@ func addArrayValue(doc *tomlDoc, name string, v reflect.Value, ctx string) error
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return &EncodeError{Path: fmt.Sprintf("%s[%d]", joinKey(ctx, name), i), Err: err}
|
return &EncodeError{Path: fmt.Sprintf("%s[%d]", joinKey(ctx, name), i), Err: err}
|
||||||
}
|
}
|
||||||
|
if mv == nil {
|
||||||
|
return &EncodeError{Path: fmt.Sprintf("%s[%d]", joinKey(ctx, name), i), Err: errNilMarshalTOML}
|
||||||
|
}
|
||||||
ev = reflect.ValueOf(mv)
|
ev = reflect.ValueOf(mv)
|
||||||
ev = followPtr(ev)
|
ev = followPtr(ev)
|
||||||
}
|
}
|
||||||
@@ -450,7 +486,14 @@ func normaliseValue(v reflect.Value) (any, error) {
|
|||||||
}
|
}
|
||||||
if v.CanInterface() {
|
if v.CanInterface() {
|
||||||
if m, ok := v.Interface().(Marshaler); ok {
|
if m, ok := v.Interface().(Marshaler); ok {
|
||||||
return m.MarshalTOML()
|
mv, err := m.MarshalTOML()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if mv == nil {
|
||||||
|
return nil, errNilMarshalTOML
|
||||||
|
}
|
||||||
|
return mv, nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// The datetime structs are TOML scalars; the emitter renders each of them.
|
// The datetime structs are TOML scalars; the emitter renders each of them.
|
||||||
@@ -579,7 +622,9 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
|
|||||||
path := append(append([]string{}, prefix...), t.key)
|
path := append(append([]string{}, prefix...), t.key)
|
||||||
e.writeBlankLine()
|
e.writeBlankLine()
|
||||||
e.buf.WriteByte('[')
|
e.buf.WriteByte('[')
|
||||||
writeKeyPath(&e.buf, path)
|
if err := e.writeKeyPath(path); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
e.buf.WriteString("]\n")
|
e.buf.WriteString("]\n")
|
||||||
if err := e.emitDoc(t.doc, path); err != nil {
|
if err := e.emitDoc(t.doc, path); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -590,7 +635,9 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
|
|||||||
for _, sub := range a.docs {
|
for _, sub := range a.docs {
|
||||||
e.writeBlankLine()
|
e.writeBlankLine()
|
||||||
e.buf.WriteString("[[")
|
e.buf.WriteString("[[")
|
||||||
writeKeyPath(&e.buf, path)
|
if err := e.writeKeyPath(path); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
e.buf.WriteString("]]\n")
|
e.buf.WriteString("]]\n")
|
||||||
if err := e.emitDoc(sub, path); err != nil {
|
if err := e.emitDoc(sub, path); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -615,7 +662,9 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
|
|||||||
path := append(append([]string{}, prefix...), ent.key)
|
path := append(append([]string{}, prefix...), ent.key)
|
||||||
e.writeBlankLine()
|
e.writeBlankLine()
|
||||||
e.buf.WriteByte('[')
|
e.buf.WriteByte('[')
|
||||||
writeKeyPath(&e.buf, path)
|
if err := e.writeKeyPath(path); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
e.buf.WriteString("]\n")
|
e.buf.WriteString("]\n")
|
||||||
if err := e.emitDoc(ent.doc, path); err != nil {
|
if err := e.emitDoc(ent.doc, path); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -625,7 +674,9 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
|
|||||||
for _, sub := range ent.docs {
|
for _, sub := range ent.docs {
|
||||||
e.writeBlankLine()
|
e.writeBlankLine()
|
||||||
e.buf.WriteString("[[")
|
e.buf.WriteString("[[")
|
||||||
writeKeyPath(&e.buf, path)
|
if err := e.writeKeyPath(path); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
e.buf.WriteString("]]\n")
|
e.buf.WriteString("]]\n")
|
||||||
if err := e.emitDoc(sub, path); err != nil {
|
if err := e.emitDoc(sub, path); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -637,10 +688,9 @@ func (e *encoder) emitDoc(doc *tomlDoc, prefix []string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (e *encoder) writeKV(key string, val any) error {
|
func (e *encoder) writeKV(key string, val any) error {
|
||||||
if !utf8.ValidString(key) {
|
if err := e.writeKey(key); err != nil {
|
||||||
return fmt.Errorf("interpres: key %q is not valid UTF-8", key)
|
return err
|
||||||
}
|
}
|
||||||
e.writeKey(key)
|
|
||||||
e.buf.WriteString(" = ")
|
e.buf.WriteString(" = ")
|
||||||
if err := e.writeValue(val); err != nil {
|
if err := e.writeValue(val); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -649,25 +699,30 @@ func (e *encoder) writeKV(key string, val any) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeKeyPath(buf *bytes.Buffer, path []string) {
|
func (e *encoder) writeKeyPath(path []string) error {
|
||||||
for i, p := range path {
|
for i, p := range path {
|
||||||
if i > 0 {
|
if i > 0 {
|
||||||
buf.WriteByte('.')
|
e.buf.WriteByte('.')
|
||||||
}
|
}
|
||||||
if isBareKey(p) {
|
if err := e.writeKey(p); err != nil {
|
||||||
buf.WriteString(p)
|
return err
|
||||||
continue
|
|
||||||
}
|
}
|
||||||
writeQuotedString(buf, p)
|
|
||||||
}
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (e *encoder) writeKey(key string) {
|
// writeKey writes one key, bare when it qualifies and quoted otherwise. A key
|
||||||
|
// that is not valid UTF-8 is an error; writing it anyway would emit corrupt
|
||||||
|
// TOML, because the quoted form has no representation for it.
|
||||||
|
func (e *encoder) writeKey(key string) error {
|
||||||
if isBareKey(key) {
|
if isBareKey(key) {
|
||||||
e.buf.WriteString(key)
|
e.buf.WriteString(key)
|
||||||
return
|
return nil
|
||||||
}
|
}
|
||||||
writeQuotedString(&e.buf, key)
|
if !utf8.ValidString(key) {
|
||||||
|
return fmt.Errorf("interpres: key %q is not valid UTF-8", key)
|
||||||
|
}
|
||||||
|
return writeQuotedString(&e.buf, key)
|
||||||
}
|
}
|
||||||
|
|
||||||
// writeQuotedString writes s as a TOML basic string (double-quoted) to buf.
|
// writeQuotedString writes s as a TOML basic string (double-quoted) to buf.
|
||||||
@@ -787,7 +842,9 @@ func (e *encoder) writeInlineTable(m map[string]any) error {
|
|||||||
if i > 0 {
|
if i > 0 {
|
||||||
e.buf.WriteString(", ")
|
e.buf.WriteString(", ")
|
||||||
}
|
}
|
||||||
e.writeKey(k)
|
if err := e.writeKey(k); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
e.buf.WriteString(" = ")
|
e.buf.WriteString(" = ")
|
||||||
if err := e.writeValue(m[k]); err != nil {
|
if err := e.writeValue(m[k]); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -798,12 +855,40 @@ func (e *encoder) writeInlineTable(m map[string]any) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (e *encoder) writeStringVal(s string) error {
|
func (e *encoder) writeStringVal(s string) error {
|
||||||
if e.opts.literalMultilineAt > 0 && strings.ContainsRune(s, '\n') && len(s) >= e.opts.literalMultilineAt {
|
if e.opts.literalMultilineAt > 0 && strings.ContainsRune(s, '\n') &&
|
||||||
|
len(s) >= e.opts.literalMultilineAt && canBeLiteralMultiline(s) {
|
||||||
return writeLiteralMultilineString(&e.buf, s)
|
return writeLiteralMultilineString(&e.buf, s)
|
||||||
}
|
}
|
||||||
return writeQuotedString(&e.buf, s)
|
return writeQuotedString(&e.buf, s)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// canBeLiteralMultiline reports whether s can be carried verbatim by the
|
||||||
|
// literal ”'...”' form: the form has no escapes, so a run of three single
|
||||||
|
// quotes would close the delimiter early, and control characters beyond tab,
|
||||||
|
// and a carriage return outside a CRLF pair, have no representation at all.
|
||||||
|
// Anything else falls back to the escaped basic string.
|
||||||
|
func canBeLiteralMultiline(s string) bool {
|
||||||
|
if strings.Contains(s, "'''") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := 0; i < len(s); {
|
||||||
|
r, size := utf8.DecodeRuneInString(s[i:])
|
||||||
|
switch {
|
||||||
|
case r == '\t' || r == '\n':
|
||||||
|
case r == '\r':
|
||||||
|
if !strings.HasPrefix(s[i+size:], "\n") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if r < 0x20 || r == 0x7f {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
i += size
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
// writeLiteralMultilineString writes s as a TOML literal multi-line string,
|
// writeLiteralMultilineString writes s as a TOML literal multi-line string,
|
||||||
// surrounded by triple single quotes. The opening delimiter is followed by a
|
// surrounded by triple single quotes. The opening delimiter is followed by a
|
||||||
// newline that the reader trims, so we always include one. The closing
|
// newline that the reader trims, so we always include one. The closing
|
||||||
@@ -831,7 +916,8 @@ func (e *encoder) writeFloat(v float64) error {
|
|||||||
case math.IsInf(v, -1):
|
case math.IsInf(v, -1):
|
||||||
e.buf.WriteString("-inf")
|
e.buf.WriteString("-inf")
|
||||||
case v == 0:
|
case v == 0:
|
||||||
// Normalise negative zero to positive zero (TOML has no -0).
|
// Normalise negative zero to positive zero, the contract the output
|
||||||
|
// rules in the documentation state.
|
||||||
e.buf.WriteString("0.0")
|
e.buf.WriteString("0.0")
|
||||||
default:
|
default:
|
||||||
s := strconv.FormatFloat(v, 'g', -1, 64)
|
s := strconv.FormatFloat(v, 'g', -1, 64)
|
||||||
|
|||||||
+143
-1
@@ -67,7 +67,7 @@ func TestMarshalFloatSpecials(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestMarshalFloatNormalizesNegativeZero(t *testing.T) {
|
func TestMarshalFloatNormalizesNegativeZero(t *testing.T) {
|
||||||
// TOML has no -0; the emitter must normalise negative zero to "0.0".
|
// The output contract normalises negative zero to "0.0".
|
||||||
type Cfg struct {
|
type Cfg struct {
|
||||||
Z float64 `toml:"z"`
|
Z float64 `toml:"z"`
|
||||||
}
|
}
|
||||||
@@ -267,6 +267,38 @@ func TestEncoderUseLiteralMultilineThresholdZero(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestEncoderLiteralMultilineFallsBackWhenUnsafe(t *testing.T) {
|
||||||
|
// The literal form carries the value verbatim, so content it cannot
|
||||||
|
// represent must fall back to the escaped basic string instead of
|
||||||
|
// producing output that does not re-parse.
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
in string
|
||||||
|
}{
|
||||||
|
{"embedded delimiter", "before ''' after\nsecond line"},
|
||||||
|
{"control character", "a\x01b\nsecond"},
|
||||||
|
{"delete character", "a\x7fb\nsecond"},
|
||||||
|
{"lone carriage return", "first\rsecond\nthird"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
out, err := NewEncoder().UseLiteralMultiline(5).Marshal(map[string]any{"s": c.in})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: marshal: %v", c.name, err)
|
||||||
|
}
|
||||||
|
if !bytes.HasPrefix(out, []byte("s = \"")) {
|
||||||
|
t.Errorf("%s: expected the basic quoted form, got:\n%s", c.name, out)
|
||||||
|
}
|
||||||
|
re, err := Parse(out)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("%s: re-parse: %v\ndoc:\n%s", c.name, err, out)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if re["s"] != c.in {
|
||||||
|
t.Errorf("%s: round-trip changed the value: %q", c.name, re["s"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// marshalerFunc adapts a plain function value to the Marshaler interface.
|
// marshalerFunc adapts a plain function value to the Marshaler interface.
|
||||||
// Tests use it to express "this field produces this TOML value" without a
|
// Tests use it to express "this field produces this TOML value" without a
|
||||||
// dedicated struct definition.
|
// dedicated struct definition.
|
||||||
@@ -351,6 +383,75 @@ func TestMarshalerErrorPropagates(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// nilMarshalerFunc is a Marshaler whose method returns nil with no error.
|
||||||
|
type nilMarshalerFunc struct{}
|
||||||
|
|
||||||
|
func (nilMarshalerFunc) MarshalTOML() (any, error) { return nil, nil }
|
||||||
|
|
||||||
|
func TestMarshalRejectsNilMarshalerResult(t *testing.T) {
|
||||||
|
// nil has no TOML representation, so a MarshalTOML result of nil is an
|
||||||
|
// error, not a silently dropped field.
|
||||||
|
_, err := Marshal(struct {
|
||||||
|
F nilMarshalerFunc `toml:"f"`
|
||||||
|
}{})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected an error for a nil MarshalTOML result")
|
||||||
|
}
|
||||||
|
ee, ok := errors.AsType[*EncodeError](err)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("expected an *EncodeError, got %T: %v", err, err)
|
||||||
|
}
|
||||||
|
if ee.Path != "f" {
|
||||||
|
t.Fatalf("Path = %q, want %q", ee.Path, "f")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inside a value array the nil result used to reach reflection as a zero
|
||||||
|
// Value and panic.
|
||||||
|
_, err = Marshal(map[string]any{"arr": []any{1, nilMarshalerFunc{}}})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected an error for a nil MarshalTOML result in an array")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "MarshalTOML returned a nil value") {
|
||||||
|
t.Errorf("err = %v, want the nil-result message", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Two fields that resolve to one TOML key must marshal as one key, resolved
|
||||||
|
// the way the decoder resolves it, or the output would carry a duplicate key
|
||||||
|
// and never re-parse.
|
||||||
|
func TestMarshalDuplicateKeyResolvesToOneField(t *testing.T) {
|
||||||
|
type SameLevel struct {
|
||||||
|
First int `toml:"v"`
|
||||||
|
Second string `toml:"v"`
|
||||||
|
}
|
||||||
|
out, err := Marshal(SameLevel{First: 1, Second: "s"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal: %v", err)
|
||||||
|
}
|
||||||
|
if want := "v = \"s\"\n"; string(out) != want {
|
||||||
|
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
type Base struct {
|
||||||
|
Name string `toml:"name"`
|
||||||
|
}
|
||||||
|
type Embedded struct {
|
||||||
|
Base
|
||||||
|
Name string `toml:"name"`
|
||||||
|
}
|
||||||
|
out, err = Marshal(Embedded{Base: Base{Name: "inner"}, Name: "outer"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal: %v", err)
|
||||||
|
}
|
||||||
|
// The shallower field wins, matching the decoder.
|
||||||
|
if want := "name = \"outer\"\n"; string(out) != want {
|
||||||
|
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
|
||||||
|
}
|
||||||
|
if _, err := Parse(out); err != nil {
|
||||||
|
t.Errorf("re-parse: %v\ndoc:\n%s", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestMarshalEmbeddedScalarStruct(t *testing.T) {
|
func TestMarshalEmbeddedScalarStruct(t *testing.T) {
|
||||||
// A field declared directly as a scalar-struct type (here LocalDateTime)
|
// A field declared directly as a scalar-struct type (here LocalDateTime)
|
||||||
// must be encoded as a TOML scalar at the parent level, not rendered as
|
// must be encoded as a TOML scalar at the parent level, not rendered as
|
||||||
@@ -529,6 +630,36 @@ func TestMarshalMixedArrayWithInlineTable(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A []any of tables is what Parse produces for a value array of inline
|
||||||
|
// tables; it must stay in the value-array form, or the output would re-parse
|
||||||
|
// as []map[string]any and the round-trip would change the value's type.
|
||||||
|
func TestMarshalValueArrayOfTablesStaysInline(t *testing.T) {
|
||||||
|
for _, doc := range []string{
|
||||||
|
"0=[{}]",
|
||||||
|
"a = [{x = 1}, {x = 2}]\n",
|
||||||
|
"b = [{x = 1}, 2, \"three\"]\n",
|
||||||
|
} {
|
||||||
|
tree, err := Parse([]byte(doc))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: parse: %v", doc, err)
|
||||||
|
}
|
||||||
|
out, err := Marshal(tree)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: marshal: %v", doc, err)
|
||||||
|
}
|
||||||
|
if bytes.HasPrefix(out, []byte("[[")) {
|
||||||
|
t.Errorf("%s: emitted the [[header]] form for a value array:\n%s", doc, out)
|
||||||
|
}
|
||||||
|
re, err := Parse(out)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: re-parse: %v\ndoc:\n%s", doc, err, out)
|
||||||
|
}
|
||||||
|
if !tomlEqual(tree, re) {
|
||||||
|
t.Errorf("%s: round-trip changed the tree:\nwas: %#v\nnow: %#v\ndoc:\n%s", doc, tree, re, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestMarshalNestedInlineTables(t *testing.T) {
|
func TestMarshalNestedInlineTables(t *testing.T) {
|
||||||
tree := map[string]any{
|
tree := map[string]any{
|
||||||
"mix": []any{
|
"mix": []any{
|
||||||
@@ -1096,6 +1227,17 @@ func TestMarshalKeyRequiresUTF8(t *testing.T) {
|
|||||||
if _, err := Marshal(m); err == nil {
|
if _, err := Marshal(m); err == nil {
|
||||||
t.Errorf("expected error for invalid UTF-8 key")
|
t.Errorf("expected error for invalid UTF-8 key")
|
||||||
}
|
}
|
||||||
|
// The check must reach the keys of table headers and of inline tables
|
||||||
|
// nested inside value arrays, not only scalar keys: both write keys
|
||||||
|
// through the same path.
|
||||||
|
nested := map[string]any{"\xff": map[string]any{"k": "v"}}
|
||||||
|
if _, err := Marshal(nested); err == nil {
|
||||||
|
t.Errorf("expected error for invalid UTF-8 table header key")
|
||||||
|
}
|
||||||
|
inline := map[string]any{"mix": []any{1, map[string]any{"\xff": 1}}}
|
||||||
|
if _, err := Marshal(inline); err == nil {
|
||||||
|
t.Errorf("expected error for invalid UTF-8 inline table key")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestMarshalStringRequiresUTF8(t *testing.T) {
|
func TestMarshalStringRequiresUTF8(t *testing.T) {
|
||||||
|
|||||||
@@ -32,6 +32,10 @@ func FuzzParse(f *testing.F) {
|
|||||||
"x = \"unterminated\n",
|
"x = \"unterminated\n",
|
||||||
"[a]\n[a]\n",
|
"[a]\n[a]\n",
|
||||||
"n = 0x1_0000_0000_0000_0000\n",
|
"n = 0x1_0000_0000_0000_0000\n",
|
||||||
|
// TOML 1.1 forms.
|
||||||
|
"t = 13:37\ndt = 1979-05-27T07:32\nodt = 1979-05-27 07:32Z\n",
|
||||||
|
"esc = \"\\e\\x41\\x7f\\x00\"\n",
|
||||||
|
"m = {\n\ta = 1,\n\tb = [1, 2,],\n\tc = { d = 2 },\n} # close\n",
|
||||||
}
|
}
|
||||||
for _, s := range seeds {
|
for _, s := range seeds {
|
||||||
f.Add([]byte(s))
|
f.Add([]byte(s))
|
||||||
|
|||||||
+5
-2
@@ -110,7 +110,9 @@ func ParseContext(ctx context.Context, data []byte) (map[string]any, error) {
|
|||||||
if !utf8.Valid(data) {
|
if !utf8.Valid(data) {
|
||||||
return nil, &SyntaxError{Line: 1, Msg: "input is not valid UTF-8"}
|
return nil, &SyntaxError{Line: 1, Msg: "input is not valid UTF-8"}
|
||||||
}
|
}
|
||||||
p := &parser{src: []rune(string(data)), line: 1, ctx: ctx}
|
// The parser scans data in place; it only reads the buffer, and every
|
||||||
|
// string it stores in the tree is copied out of it.
|
||||||
|
p := &parser{src: data, line: 1, ctx: ctx}
|
||||||
return p.parse()
|
return p.parse()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -196,7 +198,8 @@ type Unmarshaler interface {
|
|||||||
UnmarshalTOML(data any) error
|
UnmarshalTOML(data any) error
|
||||||
}
|
}
|
||||||
|
|
||||||
// Marshal returns the TOML 1.0 encoding of v.
|
// Marshal returns the TOML encoding of v. The output stays within TOML 1.0,
|
||||||
|
// so it is valid under both TOML 1.0 and 1.1.
|
||||||
//
|
//
|
||||||
// Marshal traverses v using reflection and applies the following rules:
|
// Marshal traverses v using reflection and applies the following rules:
|
||||||
//
|
//
|
||||||
|
|||||||
+162
-1
@@ -5,6 +5,7 @@ package interpres
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"math"
|
"math"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
@@ -316,6 +317,26 @@ func TestDisallowUnknownFields(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDisallowUnknownFieldsReportsSmallestKey(t *testing.T) {
|
||||||
|
// Map iteration order is random, so the reported key must be chosen
|
||||||
|
// deterministically: the smallest unknown key, whichever order the map
|
||||||
|
// iterates in.
|
||||||
|
type C struct {
|
||||||
|
Known string `toml:"known"`
|
||||||
|
}
|
||||||
|
data := []byte("known = \"x\"\nzeta = 1\nalpha = 2\nmu = 3\n")
|
||||||
|
for range 20 {
|
||||||
|
var c C
|
||||||
|
err := NewDecoder().DisallowUnknownFields().Decode(data, &c)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error for unknown fields")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), `unknown field "alpha"`) {
|
||||||
|
t.Fatalf("err = %v, want the smallest unknown key alpha", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestSkippedFieldTag(t *testing.T) {
|
func TestSkippedFieldTag(t *testing.T) {
|
||||||
type C struct {
|
type C struct {
|
||||||
Keep string `toml:"keep"`
|
Keep string `toml:"keep"`
|
||||||
@@ -421,6 +442,10 @@ func TestAcceptsNumberEdgeCases(t *testing.T) {
|
|||||||
"3.14": 3.14,
|
"3.14": 3.14,
|
||||||
"6.022e23": 6.022e23,
|
"6.022e23": 6.022e23,
|
||||||
"1e10": 1e10,
|
"1e10": 1e10,
|
||||||
|
"1e0": 1.0,
|
||||||
|
"1e06": 1e6,
|
||||||
|
"0e00": 0.0,
|
||||||
|
"2E-3": 2e-3,
|
||||||
"-2.5E-3": -2.5e-3,
|
"-2.5E-3": -2.5e-3,
|
||||||
}
|
}
|
||||||
for tok, want := range cases {
|
for tok, want := range cases {
|
||||||
@@ -487,6 +512,9 @@ func TestRejectsInlineTableExtension(t *testing.T) {
|
|||||||
"by header": "a = { b = 1 }\n[a.c]\nx = 2\n",
|
"by header": "a = { b = 1 }\n[a.c]\nx = 2\n",
|
||||||
"by dotted key": "a = { b = 1 }\na.c = 2\n",
|
"by dotted key": "a = { b = 1 }\na.c = 2\n",
|
||||||
"header over it": "a = { b = 1 }\n[a]\nx = 2\n",
|
"header over it": "a = { b = 1 }\n[a]\nx = 2\n",
|
||||||
|
// The frozen check must cover the intermediate steps of an array-of-tables
|
||||||
|
// header, not only the leaf: [[a.b.c]] walks through a and a.b.
|
||||||
|
"by nested array header": "a = { b = {} }\n[[a.b.c]]\nx = 2\n",
|
||||||
}
|
}
|
||||||
for name, doc := range cases {
|
for name, doc := range cases {
|
||||||
if _, err := Parse([]byte(doc)); err == nil {
|
if _, err := Parse([]byte(doc)); err == nil {
|
||||||
@@ -495,6 +523,36 @@ func TestRejectsInlineTableExtension(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A new element of an array of tables starts a fresh scope: sub-table headers,
|
||||||
|
// nested arrays of tables, and dotted-key paths recorded for the previous
|
||||||
|
// element must not block the same paths in the next one.
|
||||||
|
func TestArrayOfTablesFreshScopePerElement(t *testing.T) {
|
||||||
|
cases := map[string]string{
|
||||||
|
"nested array of tables": "[[a]]\n[[a.b]]\nx = 1\n[[a]]\n[a.b]\ny = 2\n",
|
||||||
|
"dotted key": "[[a]]\nb.c = 1\n[[a]]\n[a.b]\nd = 2\n",
|
||||||
|
}
|
||||||
|
for name, doc := range cases {
|
||||||
|
tree, err := Parse([]byte(doc))
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("%s: %v", name, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
elements := tree["a"].([]map[string]any)
|
||||||
|
if len(elements) != 2 {
|
||||||
|
t.Errorf("%s: len(a) = %d, want 2", name, len(elements))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Within one element the redefinition rules keep applying.
|
||||||
|
for name, doc := range map[string]string{
|
||||||
|
"header over dotted in one element": "[[a]]\nb.c = 1\n[a.b]\nd = 2\n",
|
||||||
|
"table over nested array": "[[a]]\n[[a.b]]\n[a.b]\nx = 1\n",
|
||||||
|
} {
|
||||||
|
if _, err := Parse([]byte(doc)); err == nil {
|
||||||
|
t.Errorf("%s: expected an error, got none", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRejectsSpecInvalid(t *testing.T) {
|
func TestRejectsSpecInvalid(t *testing.T) {
|
||||||
cases := map[string]string{
|
cases := map[string]string{
|
||||||
"single-digit hour": "a = 2023-10-01T1:32:00Z\n",
|
"single-digit hour": "a = 2023-10-01T1:32:00Z\n",
|
||||||
@@ -503,7 +561,8 @@ func TestRejectsSpecInvalid(t *testing.T) {
|
|||||||
"dotted over header": "[a.b]\nx = 1\n[a]\nb.y = 2\n",
|
"dotted over header": "[a.b]\nx = 1\n[a]\nb.y = 2\n",
|
||||||
"table over array": "[[t]]\n[t]\n",
|
"table over array": "[[t]]\n[t]\n",
|
||||||
"truncated datetime": "a = 2026-01-02T\n",
|
"truncated datetime": "a = 2026-01-02T\n",
|
||||||
"datetime no seconds": "a = 2026-01-02T07:32\n",
|
// "datetime no seconds" moved to the acceptance tests: TOML 1.1
|
||||||
|
// makes the seconds optional.
|
||||||
}
|
}
|
||||||
for name, doc := range cases {
|
for name, doc := range cases {
|
||||||
if _, err := Parse([]byte(doc)); err == nil {
|
if _, err := Parse([]byte(doc)); err == nil {
|
||||||
@@ -540,3 +599,105 @@ host = "h2"
|
|||||||
t.Errorf("forms[1].smtp.host = %v", h)
|
t.Errorf("forms[1].smtp.host = %v", h)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- TOML 1.1 --------------------------------------------------------------
|
||||||
|
|
||||||
|
func TestParseAcceptsNoSecondsDatetimes(t *testing.T) {
|
||||||
|
tree, err := Parse([]byte(`t = 13:37
|
||||||
|
dt = 1979-05-27T07:32
|
||||||
|
odt1 = 1979-05-27 07:32Z
|
||||||
|
odt2 = 1979-05-27 07:32-07:00
|
||||||
|
`))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse: %v", err)
|
||||||
|
}
|
||||||
|
if got := tree["t"].(LocalTime).String(); got != "13:37:00" {
|
||||||
|
t.Errorf("t = %q, want %q", got, "13:37:00")
|
||||||
|
}
|
||||||
|
if got := tree["dt"].(LocalDateTime).String(); got != "1979-05-27T07:32:00" {
|
||||||
|
t.Errorf("dt = %q, want %q", got, "1979-05-27T07:32:00")
|
||||||
|
}
|
||||||
|
if got := tree["odt1"].(time.Time).Format(time.RFC3339Nano); got != "1979-05-27T07:32:00Z" {
|
||||||
|
t.Errorf("odt1 = %q", got)
|
||||||
|
}
|
||||||
|
if got := tree["odt2"].(time.Time).Format(time.RFC3339Nano); got != "1979-05-27T07:32:00-07:00" {
|
||||||
|
t.Errorf("odt2 = %q", got)
|
||||||
|
}
|
||||||
|
// The fraction still requires the seconds it belongs to.
|
||||||
|
if _, err := Parse([]byte("a = 07:32.5\n")); err == nil {
|
||||||
|
t.Error("07:32.5: expected an error, got none")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseAcceptsEscapeAndHexEscapes(t *testing.T) {
|
||||||
|
tree, err := Parse([]byte(`esc = "\e"
|
||||||
|
hex = "\x20\x7f\xf8"
|
||||||
|
nul = "\x00"
|
||||||
|
multi = """\x68\x65"""
|
||||||
|
lit = '\x20'
|
||||||
|
`))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse: %v", err)
|
||||||
|
}
|
||||||
|
if got := tree["esc"].(string); got != "\x1b" {
|
||||||
|
t.Errorf("esc = %q, want the escape character", got)
|
||||||
|
}
|
||||||
|
if got := tree["hex"].(string); got != " \x7f\u00f8" {
|
||||||
|
t.Errorf("hex = %q", got)
|
||||||
|
}
|
||||||
|
if got := tree["nul"].(string); got != "\x00" {
|
||||||
|
t.Errorf("nul = %q", got)
|
||||||
|
}
|
||||||
|
if got := tree["multi"].(string); got != "he" {
|
||||||
|
t.Errorf("multi = %q", got)
|
||||||
|
}
|
||||||
|
// A literal string carries the sequence verbatim.
|
||||||
|
if got := tree["lit"].(string); got != `\x20` {
|
||||||
|
t.Errorf("lit = %q, want the verbatim sequence", got)
|
||||||
|
}
|
||||||
|
// Two digits exactly; a short or non-hex escape is an error.
|
||||||
|
for _, doc := range []string{`a = "\x4"`, `a = "\x"`, `a = "\xgg"`} {
|
||||||
|
if _, err := Parse([]byte(doc)); err == nil {
|
||||||
|
t.Errorf("%s: expected an error, got none", doc)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseAcceptsMultilineInlineTables(t *testing.T) {
|
||||||
|
tree, err := Parse([]byte("tbl = {\n\thello = \"world\",\n\tarr = [1,\n\t\t2,\n\t],\n\tsub = {\n\t\tk = 1,\n\t},\n\tbare = 2}\n"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse: %v", err)
|
||||||
|
}
|
||||||
|
tbl := tree["tbl"].(map[string]any)
|
||||||
|
if tbl["hello"] != "world" || tbl["bare"] != int64(2) {
|
||||||
|
t.Fatalf("tbl = %#v", tbl)
|
||||||
|
}
|
||||||
|
if arr := tbl["arr"].([]any); len(arr) != 2 {
|
||||||
|
t.Errorf("arr = %#v", tbl["arr"])
|
||||||
|
}
|
||||||
|
if sub := tbl["sub"].(map[string]any); sub["k"] != int64(1) {
|
||||||
|
t.Errorf("sub = %#v", tbl["sub"])
|
||||||
|
}
|
||||||
|
// Comments inside the table, and a trailing comma at both depths.
|
||||||
|
tree, err = Parse([]byte("m = { # one\n\t# two\n\ta = 1, # three\n\t# four\n}\n"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parse with comments: %v", err)
|
||||||
|
}
|
||||||
|
if m := tree["m"].(map[string]any); m["a"] != int64(1) {
|
||||||
|
t.Errorf("m = %#v", m)
|
||||||
|
}
|
||||||
|
// The old single-line shapes keep working, with and without the comma.
|
||||||
|
if _, err := Parse([]byte("a = { b = 1, c = 2 }\n")); err != nil {
|
||||||
|
t.Errorf("single line: %v", err)
|
||||||
|
}
|
||||||
|
// Still rejected: two commas, a missing value, and an unclosed table.
|
||||||
|
for name, doc := range map[string]string{
|
||||||
|
"double comma": "a = { b = 1,, c = 2 }\n",
|
||||||
|
"missing value": "a = {\n\tb =\n}\n",
|
||||||
|
"unterminated": "a = { b = 1,\n",
|
||||||
|
} {
|
||||||
|
if _, err := Parse([]byte(doc)); err == nil {
|
||||||
|
t.Errorf("%s: expected an error, got none", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -94,7 +94,7 @@ dev:
|
|||||||
|
|
||||||
# Runs the official toml-test compliance suite against the built adapter; toml-test must be on PATH (go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0); not standard because no canonical recipe covers a domain compliance suite.
|
# Runs the official toml-test compliance suite against the built adapter; toml-test must be on PATH (go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0); not standard because no canonical recipe covers a domain compliance suite.
|
||||||
toml-test: build
|
toml-test: build
|
||||||
toml-test test -decoder=bin/interpres-decode -toml=1.0
|
toml-test test -decoder=bin/interpres-decode -toml=1.1
|
||||||
|
|
||||||
# Coverage report as an HTML map from the gate's profile; not standard because the gate needs only the numeric floor, and a browser artefact is exploration, not a gate.
|
# Coverage report as an HTML map from the gate's profile; not standard because the gate needs only the numeric floor, and a browser artefact is exploration, not a gate.
|
||||||
coverage-html: test
|
coverage-html: test
|
||||||
|
|||||||
@@ -94,26 +94,43 @@ func decodeFloat(tok string) (any, error) {
|
|||||||
if err := checkNoLeadingZero(ip); err != nil {
|
if err := checkNoLeadingZero(ip); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
build := sign + ip
|
|
||||||
|
|
||||||
|
fp := ""
|
||||||
if hasDot {
|
if hasDot {
|
||||||
fp, err := joinDigits(frac, isDecDigit)
|
if fp, err = joinDigits(frac, isDecDigit); err != nil {
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
build += "." + fp
|
|
||||||
}
|
}
|
||||||
// The ABNF requires at least one digit after the exponent marker, so a
|
// The ABNF requires at least one digit after the exponent marker, so a
|
||||||
// trailing e or E is an error even though strconv would accept it.
|
// trailing e or E is an error even though strconv would accept it. The
|
||||||
|
// digits are a zero-prefixable integer, so leading zeros are fine here
|
||||||
|
// (the corpus holds valid cases such as 1e06 and 0e00).
|
||||||
|
esign, ed := "", ""
|
||||||
if hasExp {
|
if hasExp {
|
||||||
esign, edigits := splitSign(exp)
|
var digits string
|
||||||
ed, err := joinDigits(edigits, isDecDigit)
|
esign, digits = splitSign(exp)
|
||||||
if err != nil {
|
if ed, err = joinDigits(digits, isDecDigit); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
build += "e" + esign + ed
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The checks above validated the token's shape, and every character a
|
||||||
|
// valid token may carry is one strconv.ParseFloat accepts in place, so
|
||||||
|
// only a token with underscores needs the stripped rebuild.
|
||||||
|
if !strings.ContainsRune(tok, '_') {
|
||||||
|
f, err := strconv.ParseFloat(tok, 64)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("invalid float %q", tok)
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
build := sign + ip
|
||||||
|
if hasDot {
|
||||||
|
build += "." + fp
|
||||||
|
}
|
||||||
|
if hasExp {
|
||||||
|
build += "e" + esign + ed
|
||||||
|
}
|
||||||
f, err := strconv.ParseFloat(build, 64)
|
f, err := strconv.ParseFloat(build, 64)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("invalid float %q", tok)
|
return nil, fmt.Errorf("invalid float %q", tok)
|
||||||
@@ -123,11 +140,20 @@ func decodeFloat(tok string) (any, error) {
|
|||||||
|
|
||||||
// joinDigits validates that every rune is a digit (per isDigit) and that each
|
// joinDigits validates that every rune is a digit (per isDigit) and that each
|
||||||
// underscore sits between two digits, returning the digits with underscores
|
// underscore sits between two digits, returning the digits with underscores
|
||||||
// removed.
|
// removed. A token without underscores, the common case, is validated in
|
||||||
|
// place and returned without a copy.
|
||||||
func joinDigits(s string, isDigit func(byte) bool) (string, error) {
|
func joinDigits(s string, isDigit func(byte) bool) (string, error) {
|
||||||
if s == "" {
|
if s == "" {
|
||||||
return "", fmt.Errorf("number is missing digits")
|
return "", fmt.Errorf("number is missing digits")
|
||||||
}
|
}
|
||||||
|
if !strings.ContainsRune(s, '_') {
|
||||||
|
for i := range len(s) {
|
||||||
|
if !isDigit(s[i]) {
|
||||||
|
return "", fmt.Errorf("invalid character %q in number", string(s[i]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
for i := range len(s) {
|
for i := range len(s) {
|
||||||
c := s[i]
|
c := s[i]
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ctxCheckInterval is the number of top-level parser iterations between
|
// ctxCheckInterval is the number of top-level parser iterations between
|
||||||
@@ -16,8 +17,15 @@ import (
|
|||||||
const ctxCheckInterval = 64
|
const ctxCheckInterval = 64
|
||||||
|
|
||||||
// parser is a recursive-descent TOML parser producing a map[string]any tree.
|
// parser is a recursive-descent TOML parser producing a map[string]any tree.
|
||||||
|
//
|
||||||
|
// The scanner works on bytes, not runes: the input is validated UTF-8 before
|
||||||
|
// the parser runs, every character that drives the grammar (quotes,
|
||||||
|
// separators, newlines, bare-key characters) is ASCII, and multi-byte runes
|
||||||
|
// matter only as string content, where they are decoded on the spot. Holding
|
||||||
|
// the source as []rune instead would cost a conversion pass plus four bytes
|
||||||
|
// per rune of extra memory before parsing even starts.
|
||||||
type parser struct {
|
type parser struct {
|
||||||
src []rune
|
src []byte
|
||||||
pos int
|
pos int
|
||||||
line int
|
line int
|
||||||
ctx context.Context
|
ctx context.Context
|
||||||
@@ -85,10 +93,10 @@ func (p *parser) checkCtx() error {
|
|||||||
|
|
||||||
func (p *parser) parseTableHeader() error {
|
func (p *parser) parseTableHeader() error {
|
||||||
array := false
|
array := false
|
||||||
p.next() // consume '['
|
p.pos++ // consume '['
|
||||||
if !p.eof() && p.peek() == '[' {
|
if !p.eof() && p.peek() == '[' {
|
||||||
array = true
|
array = true
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
|
|
||||||
key, err := p.parseKeyPath()
|
key, err := p.parseKeyPath()
|
||||||
@@ -100,12 +108,12 @@ func (p *parser) parseTableHeader() error {
|
|||||||
if p.eof() || p.peek() != ']' {
|
if p.eof() || p.peek() != ']' {
|
||||||
return p.errf("expected ']' to close table header")
|
return p.errf("expected ']' to close table header")
|
||||||
}
|
}
|
||||||
p.next()
|
p.pos++
|
||||||
if array {
|
if array {
|
||||||
if p.eof() || p.peek() != ']' {
|
if p.eof() || p.peek() != ']' {
|
||||||
return p.errf("expected ']]' to close array-of-tables header")
|
return p.errf("expected ']]' to close array-of-tables header")
|
||||||
}
|
}
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
|
|
||||||
if array {
|
if array {
|
||||||
@@ -171,7 +179,12 @@ func (p *parser) tableAt(key []string) (map[string]any, error) {
|
|||||||
|
|
||||||
func (p *parser) appendArrayTable(key []string) (map[string]any, error) {
|
func (p *parser) appendArrayTable(key []string) (map[string]any, error) {
|
||||||
parent := p.root
|
parent := p.root
|
||||||
|
path := make([]string, 0, len(key))
|
||||||
for _, k := range key[:len(key)-1] {
|
for _, k := range key[:len(key)-1] {
|
||||||
|
path = append(path, k)
|
||||||
|
if p.frozen[pathKey(path)] {
|
||||||
|
return nil, p.errf("cannot extend inline table %q", strings.Join(path, "."))
|
||||||
|
}
|
||||||
existing, ok := parent[k]
|
existing, ok := parent[k]
|
||||||
if !ok {
|
if !ok {
|
||||||
next := map[string]any{}
|
next := map[string]any{}
|
||||||
@@ -213,7 +226,7 @@ func (p *parser) parseKeyValue() error {
|
|||||||
if p.eof() || p.peek() != '=' {
|
if p.eof() || p.peek() != '=' {
|
||||||
return p.errf("expected '=' after key")
|
return p.errf("expected '=' after key")
|
||||||
}
|
}
|
||||||
p.next()
|
p.pos++
|
||||||
p.skipInline()
|
p.skipInline()
|
||||||
|
|
||||||
val, err := p.parseValue()
|
val, err := p.parseValue()
|
||||||
@@ -222,7 +235,10 @@ func (p *parser) parseKeyValue() error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
dest := p.current
|
dest := p.current
|
||||||
abs := append([]string{}, p.currentPath...)
|
// One allocation covers the current section plus the dotted key; a
|
||||||
|
// top-level statement reuses it for the leaf.
|
||||||
|
abs := make([]string, 0, len(p.currentPath)+len(key))
|
||||||
|
abs = append(abs, p.currentPath...)
|
||||||
for _, k := range key[:len(key)-1] {
|
for _, k := range key[:len(key)-1] {
|
||||||
abs = append(abs, k)
|
abs = append(abs, k)
|
||||||
if p.frozen[pathKey(abs)] {
|
if p.frozen[pathKey(abs)] {
|
||||||
@@ -269,18 +285,17 @@ func (p *parser) freezeInline(path []string, val any) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// resetScopeUnder forgets the header and freeze records nested under key, which
|
// resetScopeUnder forgets the definition records nested under key, which
|
||||||
// belong to the previous element of an array of tables.
|
// belong to the previous element of an array of tables: headers, frozen
|
||||||
|
// inline tables, dotted-key paths, and nested arrays of tables all start
|
||||||
|
// fresh in the new element.
|
||||||
func (p *parser) resetScopeUnder(key []string) {
|
func (p *parser) resetScopeUnder(key []string) {
|
||||||
prefix := pathKey(key) + "\x00"
|
prefix := pathKey(key) + "\x00"
|
||||||
for k := range p.headers {
|
for _, m := range []map[string]bool{p.headers, p.frozen, p.dotted, p.arrays} {
|
||||||
if strings.HasPrefix(k, prefix) {
|
for k := range m {
|
||||||
delete(p.headers, k)
|
if strings.HasPrefix(k, prefix) {
|
||||||
}
|
delete(m, k)
|
||||||
}
|
}
|
||||||
for k := range p.frozen {
|
|
||||||
if strings.HasPrefix(k, prefix) {
|
|
||||||
delete(p.frozen, k)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -297,7 +312,7 @@ func (p *parser) parseKeyPath() ([]string, error) {
|
|||||||
parts = append(parts, part)
|
parts = append(parts, part)
|
||||||
p.skipInline()
|
p.skipInline()
|
||||||
if !p.eof() && p.peek() == '.' {
|
if !p.eof() && p.peek() == '.' {
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
break
|
break
|
||||||
@@ -309,7 +324,7 @@ func (p *parser) parseKeyComponent() (string, error) {
|
|||||||
if p.eof() {
|
if p.eof() {
|
||||||
return "", p.errf("expected a key")
|
return "", p.errf("expected a key")
|
||||||
}
|
}
|
||||||
switch c := p.peek(); c {
|
switch p.peek() {
|
||||||
case '"':
|
case '"':
|
||||||
if p.lookahead(`"""`) {
|
if p.lookahead(`"""`) {
|
||||||
return "", p.errf("multiline strings are not allowed in keys")
|
return "", p.errf("multiline strings are not allowed in keys")
|
||||||
@@ -326,13 +341,14 @@ func (p *parser) parseKeyComponent() (string, error) {
|
|||||||
c := p.peek()
|
c := p.peek()
|
||||||
if (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') ||
|
if (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') ||
|
||||||
(c >= '0' && c <= '9') || c == '_' || c == '-' {
|
(c >= '0' && c <= '9') || c == '_' || c == '-' {
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
if p.pos == start {
|
if p.pos == start {
|
||||||
return "", p.errf("invalid key character %q", string(p.peek()))
|
r, _ := utf8.DecodeRune(p.src[p.pos:])
|
||||||
|
return "", p.errf("invalid key character %q", string(r))
|
||||||
}
|
}
|
||||||
return string(p.src[start:p.pos]), nil
|
return string(p.src[start:p.pos]), nil
|
||||||
}
|
}
|
||||||
@@ -381,7 +397,7 @@ func (p *parser) parseAtom() (any, error) {
|
|||||||
// A date may be followed by a space and a time, forming one date-time.
|
// A date may be followed by a space and a time, forming one date-time.
|
||||||
if isDateToken(tok) && !p.eof() && p.peek() == ' ' {
|
if isDateToken(tok) && !p.eof() && p.peek() == ' ' {
|
||||||
if next, ok := p.peekAt(1); ok && next >= '0' && next <= '9' {
|
if next, ok := p.peekAt(1); ok && next >= '0' && next <= '9' {
|
||||||
p.next() // consume the separating space
|
p.pos++ // consume the separating space
|
||||||
timeStart := p.pos
|
timeStart := p.pos
|
||||||
p.scanBareToken()
|
p.scanBareToken()
|
||||||
tok = tok + " " + string(p.src[timeStart:p.pos])
|
tok = tok + " " + string(p.src[timeStart:p.pos])
|
||||||
@@ -406,7 +422,7 @@ func (p *parser) scanBareToken() {
|
|||||||
c == ',' || c == ']' || c == '}' || c == '#' {
|
c == ',' || c == ']' || c == '}' || c == '#' {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -416,31 +432,32 @@ func (p *parser) parseBasicString() (string, error) {
|
|||||||
if p.lookahead(`"""`) {
|
if p.lookahead(`"""`) {
|
||||||
return p.parseMultilineString('"', true)
|
return p.parseMultilineString('"', true)
|
||||||
}
|
}
|
||||||
p.next() // opening quote
|
p.pos++ // opening quote
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
for {
|
for {
|
||||||
if p.eof() {
|
if p.eof() {
|
||||||
return "", p.errf("unterminated string")
|
return "", p.errf("unterminated string")
|
||||||
}
|
}
|
||||||
c := p.next()
|
c := p.peek()
|
||||||
switch c {
|
switch c {
|
||||||
case '"':
|
case '"':
|
||||||
|
p.pos++
|
||||||
return b.String(), nil
|
return b.String(), nil
|
||||||
case '\n':
|
case '\n':
|
||||||
return "", p.errf("unterminated string")
|
return "", p.errf("unterminated string")
|
||||||
case '\r':
|
case '\r':
|
||||||
return "", p.errf("bare carriage return is not allowed in a string")
|
return "", p.errf("bare carriage return is not allowed in a string")
|
||||||
case '\\':
|
case '\\':
|
||||||
|
p.pos++
|
||||||
r, err := p.readEscape()
|
r, err := p.readEscape()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", err
|
return "", err
|
||||||
}
|
}
|
||||||
b.WriteRune(r)
|
b.WriteRune(r)
|
||||||
default:
|
default:
|
||||||
if isControlRune(c) {
|
if err := p.writeContentRune(&b); err != nil {
|
||||||
return "", p.errf("control character U+%04X is not allowed in a string", c)
|
return "", err
|
||||||
}
|
}
|
||||||
b.WriteRune(c)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -449,38 +466,58 @@ func (p *parser) parseLiteralString() (string, error) {
|
|||||||
if p.lookahead(`'''`) {
|
if p.lookahead(`'''`) {
|
||||||
return p.parseMultilineString('\'', false)
|
return p.parseMultilineString('\'', false)
|
||||||
}
|
}
|
||||||
p.next() // opening quote
|
p.pos++ // opening quote
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
for {
|
for {
|
||||||
if p.eof() {
|
if p.eof() {
|
||||||
return "", p.errf("unterminated literal string")
|
return "", p.errf("unterminated literal string")
|
||||||
}
|
}
|
||||||
c := p.next()
|
c := p.peek()
|
||||||
if c == '\'' {
|
switch c {
|
||||||
|
case '\'':
|
||||||
|
p.pos++
|
||||||
return b.String(), nil
|
return b.String(), nil
|
||||||
}
|
case '\n':
|
||||||
if c == '\n' {
|
|
||||||
return "", p.errf("unterminated literal string")
|
return "", p.errf("unterminated literal string")
|
||||||
}
|
case '\r':
|
||||||
if c == '\r' {
|
|
||||||
return "", p.errf("bare carriage return is not allowed in a string")
|
return "", p.errf("bare carriage return is not allowed in a string")
|
||||||
|
default:
|
||||||
|
if err := p.writeContentRune(&b); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if isControlRune(c) {
|
|
||||||
return "", p.errf("control character U+%04X is not allowed in a string", c)
|
|
||||||
}
|
|
||||||
b.WriteRune(c)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *parser) parseMultilineString(quote rune, escapes bool) (string, error) {
|
// writeContentRune appends the rune at the cursor to b and advances past it.
|
||||||
|
// An ASCII byte, which includes every control character the grammar forbids,
|
||||||
|
// is checked and written directly; a multi-byte rune is decoded and can never
|
||||||
|
// be a control character.
|
||||||
|
func (p *parser) writeContentRune(b *strings.Builder) error {
|
||||||
|
c := p.peek()
|
||||||
|
if c < utf8.RuneSelf {
|
||||||
|
if isControlRune(rune(c)) {
|
||||||
|
return p.errf("control character U+%04X is not allowed in a string", c)
|
||||||
|
}
|
||||||
|
p.pos++
|
||||||
|
b.WriteByte(c)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
r, size := utf8.DecodeRune(p.src[p.pos:])
|
||||||
|
p.pos += size
|
||||||
|
b.WriteRune(r)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *parser) parseMultilineString(quote byte, escapes bool) (string, error) {
|
||||||
p.skipN(3) // opening delimiter
|
p.skipN(3) // opening delimiter
|
||||||
// A newline immediately after the opening delimiter is trimmed.
|
// A newline immediately after the opening delimiter is trimmed.
|
||||||
if !p.eof() && p.peek() == '\r' {
|
if !p.eof() && p.peek() == '\r' {
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
if !p.eof() && p.peek() == '\n' {
|
if !p.eof() && p.peek() == '\n' {
|
||||||
p.line++
|
p.line++
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
|
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
@@ -500,31 +537,32 @@ func (p *parser) parseMultilineString(quote rune, escapes bool) (string, error)
|
|||||||
return "", p.errf("too many '%c' before the closing delimiter", quote)
|
return "", p.errf("too many '%c' before the closing delimiter", quote)
|
||||||
}
|
}
|
||||||
for range n - 3 {
|
for range n - 3 {
|
||||||
b.WriteRune(quote)
|
b.WriteByte(quote)
|
||||||
}
|
}
|
||||||
p.skipN(n)
|
p.skipN(n)
|
||||||
return b.String(), nil
|
return b.String(), nil
|
||||||
}
|
}
|
||||||
for range n {
|
for range n {
|
||||||
b.WriteRune(quote)
|
b.WriteByte(quote)
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
c := p.next()
|
c := p.peek()
|
||||||
if c == '\n' {
|
switch {
|
||||||
|
case c == '\n':
|
||||||
p.line++
|
p.line++
|
||||||
b.WriteRune(c)
|
p.pos++
|
||||||
continue
|
b.WriteByte(c)
|
||||||
}
|
case c == '\r':
|
||||||
if c == '\r' {
|
if p.pos+1 < len(p.src) && p.src[p.pos+1] == '\n' {
|
||||||
if !p.eof() && p.peek() == '\n' {
|
b.WriteByte(c)
|
||||||
b.WriteRune(c)
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
return "", p.errf("bare carriage return is not allowed in a string")
|
return "", p.errf("bare carriage return is not allowed in a string")
|
||||||
}
|
case escapes && c == '\\':
|
||||||
if escapes && c == '\\' {
|
p.pos++
|
||||||
// Line-ending backslash trims the following whitespace/newlines.
|
// Line-ending backslash trims the following whitespace/newlines.
|
||||||
if p.trimLineEndingBackslash() {
|
if p.trimLineEndingBackslash() {
|
||||||
continue
|
continue
|
||||||
@@ -534,12 +572,11 @@ func (p *parser) parseMultilineString(quote rune, escapes bool) (string, error)
|
|||||||
return "", err
|
return "", err
|
||||||
}
|
}
|
||||||
b.WriteRune(r)
|
b.WriteRune(r)
|
||||||
continue
|
default:
|
||||||
|
if err := p.writeContentRune(&b); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if isControlRune(c) {
|
|
||||||
return "", p.errf("control character U+%04X is not allowed in a string", c)
|
|
||||||
}
|
|
||||||
b.WriteRune(c)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -551,7 +588,7 @@ func (p *parser) trimLineEndingBackslash() bool {
|
|||||||
for !p.eof() {
|
for !p.eof() {
|
||||||
c := p.peek()
|
c := p.peek()
|
||||||
if c == ' ' || c == '\t' || c == '\r' {
|
if c == ' ' || c == '\t' || c == '\r' {
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if c == '\n' {
|
if c == '\n' {
|
||||||
@@ -570,11 +607,11 @@ func (p *parser) trimLineEndingBackslash() bool {
|
|||||||
c := p.peek()
|
c := p.peek()
|
||||||
if c == '\n' {
|
if c == '\n' {
|
||||||
p.line++
|
p.line++
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if c == ' ' || c == '\t' || c == '\r' {
|
if c == ' ' || c == '\t' || c == '\r' {
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
break
|
break
|
||||||
@@ -598,16 +635,25 @@ func (p *parser) readEscape() (rune, error) {
|
|||||||
return '\f', nil
|
return '\f', nil
|
||||||
case 'r':
|
case 'r':
|
||||||
return '\r', nil
|
return '\r', nil
|
||||||
|
case 'e':
|
||||||
|
// TOML 1.1: the escape character.
|
||||||
|
return '\x1b', nil
|
||||||
case '"':
|
case '"':
|
||||||
return '"', nil
|
return '"', nil
|
||||||
case '\\':
|
case '\\':
|
||||||
return '\\', nil
|
return '\\', nil
|
||||||
|
case 'x':
|
||||||
|
// TOML 1.1: two hex digits, code points 0x00 through 0xFF.
|
||||||
|
return p.readUnicode(2)
|
||||||
case 'u':
|
case 'u':
|
||||||
return p.readUnicode(4)
|
return p.readUnicode(4)
|
||||||
case 'U':
|
case 'U':
|
||||||
return p.readUnicode(8)
|
return p.readUnicode(8)
|
||||||
default:
|
default:
|
||||||
return 0, p.errf("invalid escape sequence \\%c", c)
|
// The byte just consumed starts a rune: the backslash before it is a
|
||||||
|
// boundary, and the input is valid UTF-8.
|
||||||
|
r, _ := utf8.DecodeRune(p.src[p.pos-1:])
|
||||||
|
return 0, p.errf("invalid escape sequence \\%c", r)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -630,17 +676,17 @@ func (p *parser) readUnicode(n int) (rune, error) {
|
|||||||
// --- arrays and inline tables ---------------------------------------------
|
// --- arrays and inline tables ---------------------------------------------
|
||||||
|
|
||||||
func (p *parser) parseArray() (any, error) {
|
func (p *parser) parseArray() (any, error) {
|
||||||
p.next() // '['
|
p.pos++ // '['
|
||||||
arr := []any{}
|
arr := []any{}
|
||||||
for {
|
for {
|
||||||
if err := p.skipArraySpace(); err != nil {
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if p.eof() {
|
if p.eof() {
|
||||||
return nil, p.errf("unterminated array")
|
return nil, p.errf("unterminated array")
|
||||||
}
|
}
|
||||||
if p.peek() == ']' {
|
if p.peek() == ']' {
|
||||||
p.next()
|
p.pos++
|
||||||
return arr, nil
|
return arr, nil
|
||||||
}
|
}
|
||||||
v, err := p.parseValue()
|
v, err := p.parseValue()
|
||||||
@@ -648,7 +694,7 @@ func (p *parser) parseArray() (any, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
arr = append(arr, v)
|
arr = append(arr, v)
|
||||||
if err := p.skipArraySpace(); err != nil {
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if p.eof() {
|
if p.eof() {
|
||||||
@@ -656,9 +702,9 @@ func (p *parser) parseArray() (any, error) {
|
|||||||
}
|
}
|
||||||
switch p.peek() {
|
switch p.peek() {
|
||||||
case ',':
|
case ',':
|
||||||
p.next()
|
p.pos++
|
||||||
case ']':
|
case ']':
|
||||||
p.next()
|
p.pos++
|
||||||
return arr, nil
|
return arr, nil
|
||||||
default:
|
default:
|
||||||
return nil, p.errf("expected ',' or ']' in array")
|
return nil, p.errf("expected ',' or ']' in array")
|
||||||
@@ -667,16 +713,23 @@ func (p *parser) parseArray() (any, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (p *parser) parseInlineTable() (any, error) {
|
func (p *parser) parseInlineTable() (any, error) {
|
||||||
p.next() // '{'
|
p.pos++ // '{'
|
||||||
tbl := map[string]any{}
|
tbl := map[string]any{}
|
||||||
assigned := map[string]bool{}
|
assigned := map[string]bool{}
|
||||||
p.skipInline()
|
// TOML 1.1 lets an inline table span lines: interior whitespace includes
|
||||||
|
// newlines and comments, and a trailing comma is allowed before the
|
||||||
|
// closing brace.
|
||||||
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
if !p.eof() && p.peek() == '}' {
|
if !p.eof() && p.peek() == '}' {
|
||||||
p.next()
|
p.pos++
|
||||||
return tbl, nil
|
return tbl, nil
|
||||||
}
|
}
|
||||||
for {
|
for {
|
||||||
p.skipInline()
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
key, err := p.parseKeyPath()
|
key, err := p.parseKeyPath()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -685,7 +738,7 @@ func (p *parser) parseInlineTable() (any, error) {
|
|||||||
if p.eof() || p.peek() != '=' {
|
if p.eof() || p.peek() != '=' {
|
||||||
return nil, p.errf("expected '=' in inline table")
|
return nil, p.errf("expected '=' in inline table")
|
||||||
}
|
}
|
||||||
p.next()
|
p.pos++
|
||||||
p.skipInline()
|
p.skipInline()
|
||||||
val, err := p.parseValue()
|
val, err := p.parseValue()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -720,15 +773,24 @@ func (p *parser) parseInlineTable() (any, error) {
|
|||||||
dest[leaf] = val
|
dest[leaf] = val
|
||||||
assigned[pathKey(path)] = true
|
assigned[pathKey(path)] = true
|
||||||
|
|
||||||
p.skipInline()
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
if p.eof() {
|
if p.eof() {
|
||||||
return nil, p.errf("unterminated inline table")
|
return nil, p.errf("unterminated inline table")
|
||||||
}
|
}
|
||||||
switch p.peek() {
|
switch p.peek() {
|
||||||
case ',':
|
case ',':
|
||||||
p.next()
|
p.pos++
|
||||||
|
if err := p.skipNestedSpace(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !p.eof() && p.peek() == '}' {
|
||||||
|
p.pos++
|
||||||
|
return tbl, nil
|
||||||
|
}
|
||||||
case '}':
|
case '}':
|
||||||
p.next()
|
p.pos++
|
||||||
return tbl, nil
|
return tbl, nil
|
||||||
default:
|
default:
|
||||||
return nil, p.errf("expected ',' or '}' in inline table")
|
return nil, p.errf("expected ',' or '}' in inline table")
|
||||||
@@ -739,12 +801,12 @@ func (p *parser) parseInlineTable() (any, error) {
|
|||||||
// --- scanning helpers ------------------------------------------------------
|
// --- scanning helpers ------------------------------------------------------
|
||||||
|
|
||||||
func (p *parser) eof() bool { return p.pos >= len(p.src) }
|
func (p *parser) eof() bool { return p.pos >= len(p.src) }
|
||||||
func (p *parser) peek() rune { return p.src[p.pos] }
|
func (p *parser) peek() byte { return p.src[p.pos] }
|
||||||
|
|
||||||
// peekAt returns the rune at offset n from the current position and whether the
|
// peekAt returns the byte at offset n from the current position and whether the
|
||||||
// offset is within the source. Use it instead of indexing p.src directly when
|
// offset is within the source. Use it instead of indexing p.src directly when
|
||||||
// the offset may sit past the end.
|
// the offset may sit past the end.
|
||||||
func (p *parser) peekAt(n int) (rune, bool) {
|
func (p *parser) peekAt(n int) (byte, bool) {
|
||||||
i := p.pos + n
|
i := p.pos + n
|
||||||
if i < 0 || i >= len(p.src) {
|
if i < 0 || i >= len(p.src) {
|
||||||
return 0, false
|
return 0, false
|
||||||
@@ -752,7 +814,7 @@ func (p *parser) peekAt(n int) (rune, bool) {
|
|||||||
return p.src[i], true
|
return p.src[i], true
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *parser) next() rune {
|
func (p *parser) next() byte {
|
||||||
c := p.src[p.pos]
|
c := p.src[p.pos]
|
||||||
p.pos++
|
p.pos++
|
||||||
return c
|
return c
|
||||||
@@ -766,49 +828,43 @@ func (p *parser) skipN(n int) {
|
|||||||
|
|
||||||
func (p *parser) match(word string) bool {
|
func (p *parser) match(word string) bool {
|
||||||
if p.lookahead(word) {
|
if p.lookahead(word) {
|
||||||
p.skipN(len([]rune(word)))
|
p.skipN(len(word))
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// lookahead reports whether s follows the cursor. Every lookahead argument in
|
||||||
|
// the grammar is ASCII, so comparing bytes is exact.
|
||||||
func (p *parser) lookahead(s string) bool {
|
func (p *parser) lookahead(s string) bool {
|
||||||
r := []rune(s)
|
return p.pos+len(s) <= len(p.src) && string(p.src[p.pos:p.pos+len(s)]) == s
|
||||||
if p.pos+len(r) > len(p.src) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
for i, c := range r {
|
|
||||||
if p.src[p.pos+i] != c {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// skipInline consumes spaces and tabs only.
|
// skipInline consumes spaces and tabs only.
|
||||||
func (p *parser) skipInline() {
|
func (p *parser) skipInline() {
|
||||||
for !p.eof() {
|
for !p.eof() {
|
||||||
if c := p.peek(); c == ' ' || c == '\t' {
|
if c := p.peek(); c == ' ' || c == '\t' {
|
||||||
p.next()
|
p.pos++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// skipArraySpace consumes whitespace, newlines, and comments inside arrays.
|
// skipNestedSpace consumes whitespace, newlines, and comments inside a value
|
||||||
func (p *parser) skipArraySpace() error {
|
// container (an array, or an inline table under TOML 1.1).
|
||||||
|
func (p *parser) skipNestedSpace() error {
|
||||||
for !p.eof() {
|
for !p.eof() {
|
||||||
switch p.peek() {
|
switch p.peek() {
|
||||||
case ' ', '\t':
|
case ' ', '\t':
|
||||||
p.next()
|
p.pos++
|
||||||
case '\r':
|
case '\r':
|
||||||
if err := p.expectCRLF(); err != nil {
|
if err := p.expectCRLF(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
case '\n':
|
case '\n':
|
||||||
p.line++
|
p.line++
|
||||||
p.next()
|
p.pos++
|
||||||
case '#':
|
case '#':
|
||||||
if err := p.skipComment(); err != nil {
|
if err := p.skipComment(); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -825,14 +881,14 @@ func (p *parser) skipBlank() error {
|
|||||||
for !p.eof() {
|
for !p.eof() {
|
||||||
switch p.peek() {
|
switch p.peek() {
|
||||||
case ' ', '\t':
|
case ' ', '\t':
|
||||||
p.next()
|
p.pos++
|
||||||
case '\r':
|
case '\r':
|
||||||
if err := p.expectCRLF(); err != nil {
|
if err := p.expectCRLF(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
case '\n':
|
case '\n':
|
||||||
p.line++
|
p.line++
|
||||||
p.next()
|
p.pos++
|
||||||
case '#':
|
case '#':
|
||||||
if err := p.skipComment(); err != nil {
|
if err := p.skipComment(); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -845,7 +901,7 @@ func (p *parser) skipBlank() error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (p *parser) skipComment() error {
|
func (p *parser) skipComment() error {
|
||||||
p.next() // consume '#'
|
p.pos++ // consume '#'
|
||||||
for !p.eof() {
|
for !p.eof() {
|
||||||
c := p.peek()
|
c := p.peek()
|
||||||
switch {
|
switch {
|
||||||
@@ -857,11 +913,11 @@ func (p *parser) skipComment() error {
|
|||||||
}
|
}
|
||||||
return p.errf("bare carriage return is not allowed")
|
return p.errf("bare carriage return is not allowed")
|
||||||
case c == '\t':
|
case c == '\t':
|
||||||
p.next()
|
p.pos++
|
||||||
case c < 0x20 || c == 0x7f:
|
case c < 0x20 || c == 0x7f:
|
||||||
return p.errf("control character U+%04X is not allowed in a comment", c)
|
return p.errf("control character U+%04X is not allowed in a comment", c)
|
||||||
default:
|
default:
|
||||||
p.next()
|
p.pos++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
@@ -871,7 +927,7 @@ func (p *parser) skipComment() error {
|
|||||||
// line feed; a bare CR is invalid.
|
// line feed; a bare CR is invalid.
|
||||||
func (p *parser) expectCRLF() error {
|
func (p *parser) expectCRLF() error {
|
||||||
if p.pos+1 < len(p.src) && p.src[p.pos+1] == '\n' {
|
if p.pos+1 < len(p.src) && p.src[p.pos+1] == '\n' {
|
||||||
p.next() // consume CR; the LF is handled by the caller
|
p.pos++ // consume CR; the LF is handled by the caller
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
return p.errf("bare carriage return is not allowed")
|
return p.errf("bare carriage return is not allowed")
|
||||||
@@ -902,10 +958,11 @@ func (p *parser) expectLineEnd() error {
|
|||||||
}
|
}
|
||||||
if p.peek() == '\n' {
|
if p.peek() == '\n' {
|
||||||
p.line++
|
p.line++
|
||||||
p.next()
|
p.pos++
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
return p.errf("unexpected %q after value", string(p.peek()))
|
r, _ := utf8.DecodeRune(p.src[p.pos:])
|
||||||
|
return p.errf("unexpected %q after value", string(r))
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *parser) errf(format string, args ...any) error {
|
func (p *parser) errf(format string, args ...any) error {
|
||||||
|
|||||||
+2
@@ -0,0 +1,2 @@
|
|||||||
|
go test fuzz v1
|
||||||
|
[]byte("0=[{}]")
|
||||||
Reference in New Issue
Block a user