1023 lines
43 KiB
Markdown
1023 lines
43 KiB
Markdown
# API
|
|
|
|
The library exports the surface below from the `sourcedock.dev/petrbalvin/interpres/v2`
|
|
package. The snippets assume:
|
|
|
|
```go
|
|
import "sourcedock.dev/petrbalvin/interpres/v2"
|
|
```
|
|
|
|
The parser implements TOML 1.1: date-times and times without seconds, the
|
|
`\e` and `\xHH` escape sequences, and multi-line inline tables with comments
|
|
and trailing commas. The encoder emits TOML 1.1.
|
|
|
|
## Functions
|
|
|
|
### `func Parse(data []byte) (*Document, error)`
|
|
|
|
Decodes a TOML document into a [Document](#documents): the values, the order the
|
|
keys were written in, whether a table was written inline, and the comments.
|
|
The values follow the mapping in the [Decoding](#decoding) section below.
|
|
Returns `*SyntaxError` on a malformed document. Input that is not valid UTF-8
|
|
is rejected with a `SyntaxError` naming the line where the invalid byte
|
|
appears, because validity is checked during the scan. Equivalent to
|
|
`ParseContext(context.Background(), data)`.
|
|
|
|
```go
|
|
doc, err := interpres.Parse([]byte("title = \"x\"\nport = 8080\n"))
|
|
tree := doc.Map()
|
|
```
|
|
|
|
### `func ParseContext(ctx context.Context, data []byte) (*Document, error)`
|
|
|
|
The cancellable variant of `Parse`. An already-cancelled context returns
|
|
`ctx.Err()` before any work. During parsing the context is checked every 64
|
|
top-level statements, so a long document aborts without running to completion.
|
|
|
|
### `func ParseMap(data []byte) (map[string]any, error)`
|
|
|
|
Decodes a TOML document into an untyped tree, the shape this package parsed
|
|
into before [Document](#documents) existed: the order of the keys and the
|
|
comments are not part of a map, so they are dropped. Use it when only the
|
|
values matter, or when the extra bookkeeping of a document is not wanted.
|
|
Equivalent to `ParseMapContext(context.Background(), data)`.
|
|
|
|
```go
|
|
tree, err := interpres.ParseMap([]byte("title = \"x\"\nport = 8080\n"))
|
|
```
|
|
|
|
### `func ParseMapContext(ctx context.Context, data []byte) (map[string]any, error)`
|
|
|
|
The cancellable variant of `ParseMap`.
|
|
|
|
### `func ParseFile(path string) (*Document, error)`
|
|
|
|
Reads the file at `path` and parses it into a [Document](#documents), the shape
|
|
`Parse` gives. Both a read failure and a parse failure come back with the file
|
|
name as their first words, wrapped so `errors.AsType` still reaches the
|
|
`SyntaxError` inside a parse failure.
|
|
|
|
```go
|
|
doc, err := interpres.ParseFile("config.toml")
|
|
```
|
|
|
|
### `func Valid(data []byte) error`
|
|
|
|
Reports whether `data` is a valid TOML document: `nil` when the parser accepts
|
|
it, the parse error when it does not. It is the library call the `-validate`
|
|
mode of interpres-decode is built on.
|
|
|
|
```go
|
|
if err := interpres.Valid(data); err != nil {
|
|
fmt.Println("invalid:", err)
|
|
}
|
|
```
|
|
|
|
### Options
|
|
|
|
The decode and encode calls take variadic options, the shape
|
|
encoding/json/v2 uses for its own. Each is a function value over the private
|
|
settings of one call, and they compose by listing:
|
|
|
|
```go
|
|
cfg, err := interpres.Unmarshal(data, &cfg2,
|
|
interpres.RejectUnknownFields(true),
|
|
interpres.NumbersAsLiterals(true))
|
|
```
|
|
|
|
Decode options:
|
|
|
|
| Option | Default | Effect |
|
|
|---|---|---|
|
|
| `RejectUnknownFields(v bool)` | off | a key with no matching struct field is an error |
|
|
| `NumbersAsLiterals(v bool)` | off | integers and floats decode into `Number`, which carries the literal; see [Numbers as literals](#numbers-as-literals) |
|
|
| `MaxNestingDepth(depth int)` | `10000` | bound how deeply arrays and inline tables may nest |
|
|
| `MaxInputSize(size int)` | no limit | bound the size of the document, in bytes |
|
|
| `LocalTimeLocation(loc)` | nil | the zone a local date-time is carried in when it decodes into a `time.Time` |
|
|
|
|
Encode options:
|
|
|
|
| Option | Default | Effect |
|
|
|---|---|---|
|
|
| `Layout(kind LayoutKind)` | `LayoutKindGrouped` | group entries as scalars, then sub-tables, then arrays of tables; `LayoutKindDeclaration` preserves declaration order |
|
|
| `OmitEmptyArrays(v bool)` | off | skip `key = []` for empty scalar arrays |
|
|
| `LiteralMultiline(threshold int)` | `0` | emit multi-line strings of at least `threshold` bytes as literal `'''...'''` |
|
|
| `InlineTables(threshold int)` | `0` | write a sub-table inline when its single-line form is at most `threshold` bytes |
|
|
| `EmitFieldComments(v bool)` | off | print the `comment=` tag option of a field above its line or header |
|
|
|
|
### `func ParseAs[T any](data []byte) (T, error)`
|
|
|
|
The generic shorthand for `Unmarshal` with a destination variable:
|
|
|
|
```go
|
|
cfg, err := interpres.ParseAs[Config](data)
|
|
```
|
|
|
|
The zero `T` comes back with the error.
|
|
|
|
### `func NewSchema[T any]()`
|
|
|
|
Precompiles the codec for `T`: the struct schema both directions walk and the
|
|
interface flags the decoder and encoder resolve through are built once and
|
|
cached, so the first document pays the cost instead of the hot path. A `T`
|
|
that is not a struct warms nothing.
|
|
|
|
### `func Statements(r io.Reader) iter.Seq2[Statement, error]`
|
|
|
|
Iterates the top-level statements of the document r carries, in written
|
|
order: key/value statements, a `[table]` header as one statement carrying
|
|
its `Table` node, and an `[[array of tables]]` as one statement per element
|
|
with the element's node and its `Index`. Iteration stops at the first error
|
|
and at a false yield, so a caller looking for one section reads no further.
|
|
The reader is consumed in full before the first yield, because the parser
|
|
scans the source in place.
|
|
|
|
```go
|
|
for stmt, err := range interpres.Statements(file) {
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if stmt.Table != nil {
|
|
fmt.Println(stmt.Key, stmt.Table.Keys())
|
|
}
|
|
}
|
|
```
|
|
|
|
## Documents
|
|
|
|
`Parse` returns a `Document`: the value tree together with what a map cannot
|
|
carry, which is the order the keys were written in, whether a table was written
|
|
as an inline table or under a header, and the comments. `ParseMap` gives the
|
|
plain tree when none of that is wanted.
|
|
|
|
```go
|
|
doc, err := interpres.Parse(data)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
root := doc.Root()
|
|
for _, key := range root.Keys() { // written order, not sorted
|
|
entry, _ := root.Get(key)
|
|
fmt.Println(key, entry.Value())
|
|
}
|
|
```
|
|
|
|
The values are shared with the tree `ParseMap` returns, so a value read from a
|
|
document and from `doc.Map()` is the same value.
|
|
|
|
| Type | Meaning |
|
|
|---|---|
|
|
| `Document` | the parsed document: `Root()` for the top-level table, `Map()` for the value tree, `Footer()` for a comment block at the end |
|
|
| `Table` | one TOML table: `Keys()` and `Entries()` in written order, `Get(key)`, `Values()` for its part of the value tree, `Inline()` |
|
|
| `Entry` | one key: `Value()`, `Inline()`, `Table()` when the value is a table, `Elements()` for the tables of an array value |
|
|
|
|
`Elements()` holds one node per element of an array value: the tables of an
|
|
array of tables, and the inline tables inside a value array, with `nil` for the
|
|
elements that are not tables.
|
|
|
|
### Editing a document
|
|
|
|
The document is writable, which makes the read-change-write loop a round trip
|
|
through one value. The typed getters read with one call:
|
|
|
|
| Getter | Returns |
|
|
|---|---|
|
|
| `GetString(key)` | `(string, bool)` |
|
|
| `GetInt(key)` | `(int64, bool)` |
|
|
| `GetFloat(key)` | `(float64, bool)` |
|
|
| `GetBool(key)` | `(bool, bool)` |
|
|
| `GetArray(key)` | `([]any, bool)` |
|
|
| `GetTable(key)` | `(*Table, bool)` |
|
|
|
|
`Set(key, value)` stores a value, keeping an existing key's position and
|
|
comments and appending a new key to the end; a `map[string]any` value becomes
|
|
a table of its own under a header, its keys in sorted order. `Delete(key)`
|
|
removes a key and everything it holds. Every method exists on `Document` for
|
|
the root table and on `Table` for the table itself.
|
|
|
|
`Marshal` writes the document back as it stands: keys in written order, the
|
|
comments above the lines and headers they belonged to, tables that were
|
|
written inline written inline again. `UnmarshalDocument(doc, v)` decodes the
|
|
edited document into a typed destination without parsing again.
|
|
|
|
```go
|
|
doc, err := interpres.Parse(data)
|
|
doc.Set("port", 9090)
|
|
out, err := interpres.Marshal(doc)
|
|
```
|
|
|
|
### Comments
|
|
|
|
A comment belongs to the line it precedes or follows, and to the node that line
|
|
introduced:
|
|
|
|
| Written | Carried by |
|
|
|---|---|
|
|
| lines above a key | that key's `Entry`, through `Comments()` |
|
|
| a comment beside a key | that key's `Entry`, through `Trailing()` |
|
|
| lines above a `[header]` or `[[header]]` | that `Table`, through `Comments()` |
|
|
| a comment beside a header | that `Table`, through `Trailing()` |
|
|
| a comment block after the last statement | the `Document`, through `Footer()` |
|
|
|
|
`SetComments` and `SetTrailing` replace them. A line carries no leading `#`
|
|
and no surrounding space, so `# note` is stored as `note` and a bare `#` as
|
|
`""`.
|
|
|
|
A `Document` is not a value to marshal: `Marshal` writes values, so it refuses
|
|
one and points at `doc.Map()`. Writing a document back, with its order and its
|
|
comments, belongs with the editing API.
|
|
|
|
### `func Unmarshal(data []byte, v any) error`
|
|
|
|
Parses `data` and stores the result in the value pointed to by `v`, typically a
|
|
pointer to a struct or to `map[string]any`. Equivalent to
|
|
`UnmarshalContext(context.Background(), data, v)`.
|
|
|
|
```go
|
|
var cfg Config
|
|
if err := interpres.Unmarshal(data, &cfg); err != nil {
|
|
return err
|
|
}
|
|
```
|
|
|
|
### `func UnmarshalContext(ctx context.Context, data []byte, v any) error`
|
|
|
|
The cancellable variant of `Unmarshal`.
|
|
|
|
### `func Marshal(v any) ([]byte, error)`
|
|
|
|
Encodes a `struct` or `map[string]V` value, or a non-nil pointer to one, into a
|
|
TOML document. The emission rules are in the [Encoding](#encoding) section
|
|
below. Equivalent to `MarshalContext(context.Background(), v)`.
|
|
|
|
```go
|
|
out, err := interpres.Marshal(cfg)
|
|
```
|
|
|
|
### `func MarshalContext(ctx context.Context, v any) ([]byte, error)`
|
|
|
|
The cancellable variant of `Marshal`. The context is checked before any work
|
|
and every 64 fields during the reflection walk.
|
|
|
|
### `func MarshalAppend(buf []byte, v any) ([]byte, error)`
|
|
|
|
Appends the TOML encoding of `v` to `buf` and returns the extended buffer, the
|
|
shape `json.MarshalAppend` has. A failed encoding leaves `buf` untouched.
|
|
|
|
## Decoding
|
|
|
|
### Value mapping
|
|
|
|
`Parse` and `Unmarshal` map TOML values to Go types as follows:
|
|
|
|
| TOML value | Go type in the parsed tree |
|
|
|---|---|
|
|
| string | `string` |
|
|
| integer | `int64` |
|
|
| float | `float64` |
|
|
| boolean | `bool` |
|
|
| offset date-time | `OffsetDateTime` |
|
|
| local date-time | `LocalDateTime` |
|
|
| local date | `LocalDate` |
|
|
| local time | `LocalTime` |
|
|
| array | `[]any` |
|
|
| table, inline table | `map[string]any` |
|
|
| array of tables | `[]map[string]any` |
|
|
|
|
When decoding into a struct, these values convert onto the destination's
|
|
concrete types: any integer or unsigned width, floats, slices, nested structs
|
|
and `map[string]T`. `NumbersAsLiterals` replaces the two numeric rows of the
|
|
table with `Number`, which keeps the literal; see
|
|
[Numbers as literals](#numbers-as-literals).
|
|
|
|
### Target constraints
|
|
|
|
`Unmarshal`, `UnmarshalRead` and `UnmarshalContext` write into a non-nil pointer:
|
|
|
|
- `*struct`, matched per the field rules below
|
|
- `*map[string]any` or `*map[string]T`, keys become map keys and values decode
|
|
into `T` recursively; a map that already holds entries is merged into, the
|
|
document's values replacing same-named keys and the rest left standing
|
|
- `*OrderedMap`, the keys fill in the order the document wrote them; see
|
|
[Ordered tables](#ordered-tables)
|
|
- `*any`, receives the whole parsed tree unchanged
|
|
|
|
Anything else returns `interpres: decode target must be a non-nil pointer`.
|
|
|
|
### Field matching
|
|
|
|
For a struct destination, a TOML key matches a field as follows:
|
|
|
|
1. The `toml:"name"` tag, using the part before any comma. The literal `-`
|
|
excludes the field.
|
|
2. Without a tag, the lower-cased field name.
|
|
3. An anonymous (embedded) field without a tag is inlined: the decoder walks
|
|
into the embedded struct and matches its own fields against the same keys,
|
|
mirroring how the encoder flattens it. A nil embedded pointer struct is
|
|
allocated on demand. An untagged embedded map receives the keys no field
|
|
claims; when a struct embeds several untagged maps, the first one
|
|
declared takes all of them and the rest stay untouched, so the rule stays
|
|
predictable.
|
|
4. The key itself is lower-cased before lookup, so the match is
|
|
case-insensitive on both sides: `DATABASEURL` matches a field named
|
|
`DatabaseUrl`.
|
|
|
|
The match is exact after lower-casing. No separator is inserted, so a TOML key
|
|
`database_url` does not match a field named `DatabaseUrl`; tag such a field
|
|
(`toml:"database_url"`) or use the lower-cased name as the key. When two
|
|
fields resolve to the same name, the shallower one wins; at equal depth, the
|
|
one declared later wins.
|
|
|
|
Unknown keys are ignored by default, landing in an untagged embedded map when
|
|
the struct has one; [Strict decoding](#strict-decoding) rejects them instead.
|
|
|
|
The tag may carry the `required` option, `toml:"host,required"`: the decode
|
|
fails with `missing required key "host"` when no key of the document resolved
|
|
to the field. The check runs after the table is read, so the other fields
|
|
carry their values whether the required one is present or not.
|
|
|
|
### Numeric conversion
|
|
|
|
The parser produces `int64` for every integer and `float64` for every float.
|
|
The decoder converts to the destination type with explicit overflow checks:
|
|
|
|
| Destination kind | Rule |
|
|
|---|---|
|
|
| `int`, `int8`, `int16`, `int32`, `int64` | the `int64` value must not overflow the destination |
|
|
| `uint`, `uint8`, `uint16`, `uint32`, `uint64` | the value must be non-negative and must not overflow the destination's own width, `uint` on a 32-bit platform included; `uint64` accepts any non-negative `int64` |
|
|
| `float32`, `float64` | copied verbatim, except that a finite value beyond the `float32` range is an overflow error rather than a silent infinity; an integer also coerces, so TOML `5` decodes into `5.0` |
|
|
| `bool`, `string` | exact kind match only, no coercion across kinds |
|
|
| `time.Time`, `OffsetDateTime` | offset date-times only; no implicit conversion to or from the local variants |
|
|
|
|
A conversion that the rules do not allow produces an error wrapped with the
|
|
offending key or index, for example `p: interpres: integer 300 overflows uint8`.
|
|
|
|
### Numbers as literals
|
|
|
|
`NumbersAsLiterals(true)` decodes every integer and float into `Number`, a
|
|
string type that carries the literal the document wrote: `0x1f`, `1_000`,
|
|
`+1.0`, `inf`. The shape is validated as strictly as ever, so `01` and `1__0`
|
|
remain parse errors; only the evaluated value is replaced by the literal. A
|
|
round trip through the value tree and `Marshal` keeps the spelling, where the
|
|
default tree normalises `0x1f` to `31` and `+1.0` to `1.0`.
|
|
|
|
```go
|
|
var tree map[string]any
|
|
err := interpres.Unmarshal(data, &tree, interpres.NumbersAsLiterals(true))
|
|
lit := tree["rate"].(interpres.Number) // "1_000"
|
|
```
|
|
|
|
A destination of a concrete kind is unaffected: an `int64` field, a `float64`
|
|
field and a `time.Duration` field take the evaluated value they always took,
|
|
and a `Number` field takes the literal. `Number.Float64` and `Number.Int64`
|
|
evaluate the literal on demand, with an error for a float asked as an integer
|
|
and for a literal that is not a valid TOML number. `Marshal` writes a `Number`
|
|
as its bare literal and rejects one that is not a valid TOML number, whether it
|
|
stands alone or inside a value array.
|
|
|
|
### Date-time values
|
|
|
|
Offset date-times decode into `OffsetDateTime`, whose embedded `time.Time` is the
|
|
instant with the offset the document wrote; a destination of the plain
|
|
`time.Time` takes the same value, so a timestamp field does not have to name the
|
|
wrapper. The local variants decode into `LocalDateTime`, `LocalDate` and
|
|
`LocalTime`, whose embedded `time.Time` is normalised to UTC (midnight UTC for a
|
|
local date, the zero date for a local time). Every kind may omit the seconds as
|
|
of TOML 1.1 (`07:32`, `1979-05-27T07:32`); such a value carries a zero second,
|
|
and the encoder writes the seconds only when the value carries them, so a
|
|
document written without seconds comes back without them. There is no implicit
|
|
conversion between the offset and local kinds; assigning one to the other is an
|
|
error. The date-time types take a bare timestamp and never a quoted string, so a
|
|
document that writes a date-time with quotes does not decode into them, and
|
|
neither `encoding.TextUnmarshaler` nor the embedded `time.Time` changes that.
|
|
|
|
`LocalTimeLocation(loc)` lets a local date-time fill a plain
|
|
`time.Time` destination as well: the wall-clock value is carried in the
|
|
location given, relabelled rather than shifted, so `07:32` in the document is
|
|
`07:32` in the zone. Without the option the wrapper types are the only
|
|
destinations a local kind fills.
|
|
|
|
### Arrays of tables
|
|
|
|
A `[[a]]` block parses into a `[]map[string]any` element of the tree. When the
|
|
destination is a slice, each element decodes into the slice's element type
|
|
(`[]struct` or `[]map[string]V`); a mismatch on one element surfaces as an
|
|
error wrapped with `[i]:` and the element index.
|
|
|
|
A value array also decodes into a fixed-size array, `[N]T`, the mirror of the
|
|
encoder's ability to encode one. The element count has to match: an array
|
|
whose length differs from `N` is an error, `interpres: cannot assign 2
|
|
elements to [3]int`, wrapped with the key path.
|
|
|
|
### Custom decoding: `Unmarshaler`
|
|
|
|
A type that wants full control of its decode implements:
|
|
|
|
```go
|
|
type Unmarshaler interface {
|
|
UnmarshalTOML(data any) error
|
|
}
|
|
```
|
|
|
|
`data` is whatever the parser produced for that key: `string`, `bool`, `int64`,
|
|
`float64`, `OffsetDateTime`, `LocalDateTime`, `LocalDate`, `LocalTime`, `[]any`, or
|
|
`map[string]any`. The method inspects the value and mutates its own receiver;
|
|
the decoder keeps whatever state the receiver stored.
|
|
|
|
The method is usually on a pointer receiver (`*T`). The decoder invokes it when
|
|
the destination type or its pointer implements the interface, so a
|
|
pointer-receiver implementation on an addressable struct field is found
|
|
automatically, and a nil pointer destination is allocated first. An error
|
|
returned from `UnmarshalTOML` halts the decode and propagates wrapped with the
|
|
key path, for example `addr: unmarshal: not a string`.
|
|
|
|
### Custom decoding: `UnmarshalerContext`
|
|
|
|
`UnmarshalerContext` is `Unmarshaler` with the decode's context handed in:
|
|
|
|
```go
|
|
type UnmarshalerContext interface {
|
|
UnmarshalTOMLContext(ctx context.Context, data any) error
|
|
}
|
|
```
|
|
|
|
A type that implements both gets `UnmarshalTOMLContext`, so a long custom
|
|
decode can abort on cancellation instead of running to completion. The
|
|
context a non-cancellable entry point carries is `context.Background`, never
|
|
nil.
|
|
|
|
### Custom decoding: `encoding.TextUnmarshaler`
|
|
|
|
A destination type that implements `encoding.TextUnmarshaler` receives a TOML
|
|
string as its text content, the rule `encoding/json` follows:
|
|
|
|
```go
|
|
func (ip *IP) UnmarshalText(text []byte) error
|
|
```
|
|
|
|
The decoder looks for the method on the destination and on its address, so a
|
|
pointer-receiver `UnmarshalText` is invoked on an addressable struct field, and
|
|
the elements of a slice destination are reached the same way. The text path
|
|
applies to TOML strings only: every other value kind keeps its own rule, so
|
|
`r = 1` does not reach a receiver that expects text. An error from
|
|
`UnmarshalText` halts the decode and propagates with the key path and the
|
|
prefix `unmarshal text:`, for example `addr: unmarshal text: not an address`.
|
|
|
|
[`UnmarshalTOML`](#custom-decoding-unmarshaler) wins over `UnmarshalText` when
|
|
a type implements both, and the four [date-time
|
|
types](#date-time-values) are excluded: a quoted string stays a string and
|
|
never becomes an `OffsetDateTime` or one of the local wrappers.
|
|
|
|
### Durations
|
|
|
|
TOML has no duration type, so `time.Duration` has a rule of its own. The
|
|
encoder writes the canonical Go form in a TOML string, `1h30m0s`, and the
|
|
decoder reads that string back with `time.ParseDuration`. A bare integer is
|
|
still the nanosecond count it has always been, so `from_int = 5400000000000`
|
|
and `from_text = "1h30m"` decode to the same duration. Text that
|
|
`time.ParseDuration` rejects, `d = "90"` among it, fails with
|
|
`interpres: invalid duration "90"`.
|
|
|
|
### Ordered tables
|
|
|
|
`OrderedMap` is a string-keyed table that remembers the order its keys were
|
|
set in, the shape a `map[string]any` cannot carry. Decoding into one fills it
|
|
in the order the document wrote the keys, and `Marshal` writes one back in
|
|
that order, where a map destination carries no order and a map source sorts
|
|
its keys. The type is a decode target on its own, in a struct field, and as
|
|
the element of an array of tables.
|
|
|
|
```go
|
|
var cfg OrderedMap
|
|
err := interpres.Unmarshal(data, &cfg)
|
|
out, err := interpres.Marshal(&cfg) // the keys come back in written order
|
|
```
|
|
|
|
The values are untyped, the shape the parser produces, so a nested table
|
|
inside an `OrderedMap` is a plain `map[string]any`; the order is kept at the
|
|
level the `OrderedMap` sits at. Inside a value array an `OrderedMap` renders
|
|
as an ordinary inline table, whose keys are sorted.
|
|
|
|
### Strict decoding
|
|
|
|
By default unknown keys are dropped silently. The `RejectUnknownFields`
|
|
option rejects them instead:
|
|
|
|
```go
|
|
err := interpres.Unmarshal(data, &cfg, interpres.RejectUnknownFields(true))
|
|
```
|
|
|
|
A typo such as `database_urls` then fails with
|
|
`interpres: unknown field "database_urls" for main.Config` instead of a silent
|
|
default-zero run. Strictness applies to every struct the decode reaches, at any
|
|
depth, including struct elements inside slices; map destinations accept every
|
|
key by nature. When several keys are unknown, the message names the smallest
|
|
one, so it does not depend on map iteration order.
|
|
|
|
### Direct decoding
|
|
|
|
For a struct destination whose type graph carries no untagged embedded map and
|
|
no custom decode hook, the decode parses straight into
|
|
the destination: the table skeleton is resolved against the struct schema while
|
|
the document scans, and no intermediate value tree is kept. Values still flow
|
|
through the ordinary assignment rules, so every conversion, hook and error the
|
|
[Decoding](#decoding) section states holds verbatim; the parity with the tree
|
|
path is pinned by a differential fuzz target that decodes every generated
|
|
document both ways and compares the results.
|
|
|
|
A document or destination the direct skeleton cannot model — an unknown table
|
|
under strictness it must sink, a hook that needs the whole parsed value, an
|
|
embedded map filler — falls back to the tree path and reruns, so the
|
|
observable behaviour is always the tree path's, exactly. Nothing changes for
|
|
`Parse`, `ParseMap` or the document API: the tree remains theirs.
|
|
|
|
### Cancellation
|
|
|
|
`ParseContext`, `UnmarshalContext` and `MarshalContext` accept a
|
|
`context.Context`. An already-cancelled context short-circuits with
|
|
`context.Canceled` before any work begins; afterwards the context is checked
|
|
every 64 top-level statements, and inside a value too: an array, an inline
|
|
table and a multi-line string check every 64 elements or lines, so one huge
|
|
value cannot hold the parse past its cancellation.
|
|
|
|
### Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Caller
|
|
participant Unmarshal as Unmarshal
|
|
participant Parser as parser
|
|
participant Decoder as decoder
|
|
Caller->>Unmarshal: data, v
|
|
Unmarshal->>Parser: ParseContext(ctx, data)
|
|
Parser-->>Unmarshal: tree or *SyntaxError
|
|
Unmarshal->>Decoder: decode(tree, reflect value)
|
|
Decoder-->>Unmarshal: nil or wrapped field error
|
|
Unmarshal-->>Caller: error
|
|
```
|
|
|
|
## Encoding
|
|
|
|
### Input constraints
|
|
|
|
`Marshal` and `MarshalWrite` accept a `struct`, a `map[string]V`, or a
|
|
non-nil pointer to one, where `V` is any value `Marshal` itself understands. A
|
|
different top-level value fails:
|
|
|
|
| Input | Error |
|
|
|---|---|
|
|
| a bare scalar or array | `interpres: top-level value must be a struct or map[string]V, got <type>` |
|
|
| a nil `any` | `interpres: cannot marshal nil value` |
|
|
| a nil pointer | `interpres: cannot marshal nil pointer` |
|
|
|
|
The encoding walk carries a nesting limit of 10000 levels, the parser's own
|
|
figure: a value that nests deeper, which cyclic data always does, is rejected
|
|
with an error that names the limit and suggests the cycle, instead of running
|
|
the stack out.
|
|
|
|
### Field matching
|
|
|
|
Struct fields become TOML keys as follows:
|
|
|
|
1. The `toml:"name"` tag, using the part before any comma. The literal `-`
|
|
skips the field.
|
|
2. Without a tag, the lower-cased field name. The key emitted for a field named
|
|
`DatabaseUrl` is `databaseurl`; tag the field to emit `database_url`.
|
|
3. An anonymous (embedded) field without a tag is inlined into the parent
|
|
table; with a tag it is a regular field under that name.
|
|
|
|
Keys that match `[A-Za-z0-9_-]+` are emitted bare, all others quoted. A
|
|
`map[string]V` emits its keys in sorted order for deterministic output, and a
|
|
nil map emits nothing.
|
|
|
|
### Tag options
|
|
|
|
The part of a `toml` tag after the first comma carries options. They shape
|
|
emission only; the decoder ignores them, so a value that round-trips keeps
|
|
its key whether the table it came from was written inline or under a header.
|
|
|
|
- `omitzero` skips the field when its value is the zero value of its type. A
|
|
type with an `IsZero() bool` method (time.Time among them) decides through
|
|
that method, so a zero `time.Time` or an all-zero struct disappears from
|
|
the output.
|
|
- `omitempty` skips the field when it holds an empty value in the
|
|
encoding/json sense: an empty string, a zero number, `false`, a nil pointer
|
|
or interface, and a nil or empty slice, array or map. This is a change of
|
|
semantics against 1.x, where only collections were covered.
|
|
- `inline` forces a struct or map field to emit as `name = {…}`, the inline
|
|
table form, instead of a header section, whatever its size; a named
|
|
embedded struct tagged this way does the same. A field holding an array of
|
|
tables is an error under `inline`, because the inline form would re-parse
|
|
as a value array and change the value's Go type.
|
|
- `comment=text` carries a comment for the field, which
|
|
`EmitFieldComments(true)` prints above the field's line or
|
|
header, each line of a multi-line text with its own `# ` marker. Go doc
|
|
comments are not visible to reflection, so the tag is the channel that
|
|
carries the text; without the encoder option the tag is ignored.
|
|
|
|
```go
|
|
type Config struct {
|
|
Host string `toml:"host,omitzero"`
|
|
Started time.Time `toml:"started,omitzero"`
|
|
Tags []string `toml:"tags,omitempty"`
|
|
Retry Retry `toml:"retry,inline"`
|
|
}
|
|
```
|
|
|
|
Options combine after the name: `toml:"name,omitempty,omitzero"` is valid, and
|
|
an unknown option is ignored.
|
|
|
|
Untagged embedded fields round-trip: the decoder inlines embedded structs and
|
|
routes unclaimed keys into an embedded map exactly where the encoder flattened
|
|
them.
|
|
|
|
### Group-by-kind layout
|
|
|
|
By default every table is emitted with its entries grouped by kind:
|
|
|
|
1. scalars (`string`, `int64`, `float64`, `bool`, `time.Time`,
|
|
`OffsetDateTime`, `LocalDateTime`, `LocalDate`, `LocalTime`)
|
|
2. sub-tables (structs and `map[string]V` values)
|
|
3. arrays of tables (`[]struct` and `[]map[string]V`)
|
|
|
|
Within each group the order follows struct field declaration order, or sorted
|
|
key order for maps. This is the only layout that reliably re-parses to the same
|
|
tree: once a `[header]` is written, later scalars at the parent level would be
|
|
parsed as keys of the sub-table.
|
|
|
|
### Preserving declaration order
|
|
|
|
`LayoutKindDeclaration` on an `Encoder` walks the entries in declaration order
|
|
instead, emitting each header immediately before its content:
|
|
|
|
```go
|
|
out, err := interpres.Marshal(cfg, interpres.Layout(interpres.LayoutKindDeclaration))
|
|
```
|
|
|
|
The output remains parseable, but a scalar declared after a sub-table lands
|
|
under that sub-table's header when the document is read back. Use this layout
|
|
for presentation only, not when the output must round-trip.
|
|
|
|
### Custom encoding: `Marshaler`
|
|
|
|
A type that wants a non-default TOML shape implements:
|
|
|
|
```go
|
|
type Marshaler interface {
|
|
MarshalTOML() (any, error)
|
|
}
|
|
```
|
|
|
|
The returned value is encoded as if it had been passed in place of the
|
|
receiver, so it may be a scalar, a slice, an array of tables, or another
|
|
struct or map, including the `Marshaler` result of another type; the encoder
|
|
recurses, and the result is normalised like any other value, so a method may
|
|
return a plain `int` or a `time.Duration`.
|
|
|
|
An error returned from `MarshalTOML` fails the marshal wrapped with the key
|
|
path, for example `interpres: server.port: bad timestamp`. A result of `nil` with
|
|
a nil error fails the same way with `MarshalTOML returned a nil value`: nil has
|
|
no TOML representation, so dropping the field silently is not an option.
|
|
|
|
The method is reached for every value the walk meets, array elements included:
|
|
an element that renders itself as a table keeps the `[[header]]` form, one that
|
|
renders itself as a scalar turns the array into a value array, and the method
|
|
runs once per element. It is looked up on the value and on its address, so a
|
|
pointer-receiver method is called for a field or an element, exactly as
|
|
`MarshalText` is.
|
|
|
|
```go
|
|
type Port int
|
|
|
|
func (p Port) MarshalTOML() (any, error) {
|
|
return int64(p), nil
|
|
}
|
|
```
|
|
|
|
### Custom encoding: `encoding.TextMarshaler`
|
|
|
|
A type that implements `encoding.TextMarshaler` is encoded as a TOML string
|
|
holding the text the method returns, which is the rule `encoding/json` follows:
|
|
|
|
```go
|
|
func (ip IP) MarshalText() ([]byte, error)
|
|
```
|
|
|
|
The encoder looks for the method on the value and on its address, so a
|
|
pointer-receiver `MarshalText` is found on a struct field of an addressable
|
|
value (pass a pointer to `Marshal`) and always on a slice element. `net.IP`,
|
|
`netip.Addr` and user types follow this rule, and a struct that implements the
|
|
interface becomes a string rather than a table. `MarshalTOML` wins when a type
|
|
implements both, the four [date-time types](#date-time-values) keep their bare
|
|
timestamp form, and text that is not valid UTF-8 is an error rather than a
|
|
replacement character.
|
|
|
|
A duration carries no text method of its own; see [Durations](#durations) for
|
|
its rule.
|
|
|
|
### Arrays
|
|
|
|
An array whose every element is a table (`[]struct`, `[]map[string]V`, after
|
|
pointer dereference) emits as an array of tables. TOML also lets one array mix
|
|
tables with scalars; such an array emits as a plain value array, with the
|
|
table elements rendered as inline tables:
|
|
|
|
```go
|
|
tree, _ := interpres.Parse([]byte(`arr = [1, {a = 2}, "x"]`))
|
|
out, _ := interpres.Marshal(tree) // arr = [1, {a = 2}, "x"]
|
|
```
|
|
|
|
A `[]any` holding only tables keeps the value-array form as well, because that
|
|
is the shape `Parse` gives a value array of inline tables; emitting it as
|
|
`[[headers]]` would re-parse as `[]map[string]any` and change the value's type
|
|
across a round-trip.
|
|
|
|
### Empty arrays
|
|
|
|
A nil slice is always omitted. An empty (length 0) array of tables is always
|
|
omitted, because TOML forbids an empty `[[a]]`. Other empty arrays emit as
|
|
`key = []` by default; `OmitEmptyArrays()` skips them as well, so
|
|
`[]string{}` is treated like a nil slice.
|
|
|
|
### Long strings
|
|
|
|
By default every string is emitted as a basic `"..."` string with the escapes
|
|
TOML requires, a newline among them as `\n`. `LiteralMultiline(threshold)`
|
|
switches strings that contain a newline and are at least `threshold` bytes long
|
|
to the literal `'''...'''` form, which carries the newlines verbatim:
|
|
|
|
```go
|
|
out, err := interpres.Marshal(cfg, interpres.LiteralMultiline(80))
|
|
```
|
|
|
|
Single-line strings keep the basic form regardless of the threshold, and a
|
|
threshold of `0` or less disables the option. A string the literal form cannot
|
|
carry verbatim (an embedded run of three single quotes, a control character
|
|
other than tab or newline, or a carriage return outside a CRLF pair) also keeps
|
|
the basic form, so the output always re-parses to the same value.
|
|
|
|
### Inline tables
|
|
|
|
A table element of a value array, and a sub-table inlined by
|
|
[`InlineTables`](#compact-documents), is written as one `{a = 1, b = 2}` line
|
|
while it fits. An inline table that would pass the hundredth column carries
|
|
newlines and a trailing comma instead, which TOML 1.1 allows:
|
|
|
|
```toml
|
|
arr = [1, {
|
|
n = 1,
|
|
name = "a value long enough to push this line well past the one hundred column limit",
|
|
}]
|
|
```
|
|
|
|
The closing brace and the entries are indented one tab per nesting level, a
|
|
nested table is measured on its own line, and the output re-parses to the same
|
|
value either way.
|
|
|
|
### Compact documents
|
|
|
|
`InlineTables(threshold)` writes a sub-table as an inline table when its
|
|
single-line rendering is at most `threshold` bytes, and as a table header
|
|
section when it is longer. A document of small tables therefore grows shorter:
|
|
|
|
```go
|
|
out, err := interpres.Marshal(cfg, interpres.InlineTables(60))
|
|
```
|
|
|
|
With `60` and a table of three short entries, the same value is written
|
|
|
|
```toml
|
|
server = {host = "127.0.0.1", port = 9090, tls = {on = false}}
|
|
```
|
|
|
|
instead of three lines under a `[server]` header and a `[server.tls]` section.
|
|
A nested sub-table takes part in the same way, and the whole option is off at
|
|
`0` or less. Two limits are deliberate. An array of tables keeps the `[[a]]`
|
|
header form, because its inline form re-parses as a value array and would change
|
|
the value's Go type. And because an inlined table is a value line, every one of
|
|
them precedes the first header of its document, so a table inlined next to a
|
|
header is not read back as part of that header's section.
|
|
|
|
### Cancellation
|
|
|
|
`MarshalContext` accepts a `context.Context`. The
|
|
context is checked before any work and every 64 fields during the reflection
|
|
walk.
|
|
|
|
### What is not preserved
|
|
|
|
The output is not byte-identical to any document that produced the value:
|
|
|
|
- comments are dropped, and whitespace inside expressions is normalised
|
|
- map keys are emitted in sorted order
|
|
- the choice between `[table]` headers and inline tables is not preserved
|
|
- strings use the basic quoted form unless the literal option above applies
|
|
- a date-time drops its zero seconds and the trailing zeros of its fraction, so
|
|
`07:32:00` is written `07:32`; both are the same value
|
|
- floats always carry a `.` or an exponent, so a float `1` is emitted as `1.0`
|
|
and stays distinguishable from the integer `1` across a round-trip; negative
|
|
zero is normalised to `0.0`
|
|
|
|
The output is guaranteed to re-parse through `Parse` into an equivalent value
|
|
tree. `Marshal` cannot encode cyclic data structures.
|
|
|
|
### Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Caller
|
|
participant Marshal as Marshal
|
|
participant Walk as reflection walk
|
|
participant Emit as emitter
|
|
Caller->>Marshal: v any
|
|
Marshal->>Walk: build tomlDoc from struct or map
|
|
Walk-->>Marshal: tomlDoc or wrapped error
|
|
Marshal->>Emit: emitDoc(doc)
|
|
Emit-->>Marshal: bytes or error
|
|
Marshal-->>Caller: bytes, error
|
|
```
|
|
|
|
## Coming from encoding/json and encoding/json/v2
|
|
|
|
The API follows the shapes encoding/json made familiar and the option style
|
|
encoding/json/v2 made current, with the differences TOML asks for:
|
|
|
|
| encoding/json or encoding/json/v2 | interpres | Notes |
|
|
|---|---|---|
|
|
| `json.Unmarshal(data, v)` | `Unmarshal(data, v)` | the same shape; the value mapping is TOML's |
|
|
| `json.Marshal(v)` | `Marshal(v)` | the same shape; the output is TOML 1.1 |
|
|
| `json.MarshalAppend(buf, v)` | `MarshalAppend(buf, v)` | the same shape, options included |
|
|
| `json.MarshalWrite(w, v)` | `MarshalWrite(w, v)` | the same shape, options included |
|
|
| `json.UnmarshalRead(r, v)` | `UnmarshalRead(r, v)` | the same shape, options included |
|
|
| `json/v2 RejectUnknownMembers` | `RejectUnknownFields(true)` | the same effect under TOML vocabulary |
|
|
| `(*json.Decoder).DisallowUnknownFields` | `RejectUnknownFields(true)` | the variadic option replaces the stateful decoder |
|
|
| `json.Number`, `StringifyNumbers` | `Number`, `NumbersAsLiterals(true)` | the TOML literal carries its radix and separators, so `0x1f` stays `0x1f` |
|
|
| `json/v2 MarshalOptions` fields | `MarshalOption` values | `Layout`, `OmitEmptyArrays`, `LiteralMultiline`, `InlineTables`, `EmitFieldComments` |
|
|
| `json/v2 JoinOptions` | listing | options compose by listing them in the call |
|
|
| `json.MarshalIndent` | none | TOML is the presentation format; the `-json` mode of interpres-decode prints plain JSON |
|
|
| tag `json:"name,omitempty"` | tag `toml:"name,omitempty"` | the empty-value rules match encoding/json as of 2.0 |
|
|
| tag `json:"name,omitzero"` | tag `toml:"name,omitzero"` | the same, `IsZero()` honoured |
|
|
| tag `json:"name,inline"` (v2) | tag `toml:"name,inline"` | forces the inline table form on encode |
|
|
| `json/v2 Marshalers` | `Marshaler` (`MarshalTOML`) | the TOML method returns a value the encoder renders, not bytes |
|
|
| `json/v2 Unmarshalers` | `Unmarshaler` (`UnmarshalTOML`) | the data arrives decoded, not as bytes |
|
|
| `encoding.TextMarshaler`, `TextUnmarshaler` | honoured, the same | a type that renders itself as text becomes a TOML string, both ways |
|
|
| `*json.UnmarshalTypeError` | `*DecodeError` | the path is segments with a `String()` renderer, not a dotted string |
|
|
| `*json.SyntaxError` | `*SyntaxError` | the TOML error adds the byte `Offset` and the `Column` to the line |
|
|
| context support | `*Context` variants of every entry point | encoding/json has none |
|
|
|
|
## Types
|
|
|
|
### `type SyntaxError struct{ Line, Offset, Column int; Msg string }`
|
|
|
|
Describes a document the parser rejected: the 1-based `Line` at which it gave
|
|
up, the `Offset` in bytes the scan stopped at, the 1-based `Column` on that
|
|
line, and `Error()` rendering as `interpres: line N: msg`. A malformed document
|
|
is the usual cause; the nesting limit and an input that is not valid UTF-8
|
|
report through the same type, with the UTF-8 message naming the offset of the
|
|
first invalid byte. Read the structured fields with a type assertion or
|
|
`errors.AsType`:
|
|
|
|
```go
|
|
if se, ok := errors.AsType[*interpres.SyntaxError](err); ok {
|
|
fmt.Println(se.Line, se.Offset, se.Column, se.Msg)
|
|
fmt.Println(se.SourceLine(data)) // the line, with a caret under Offset
|
|
}
|
|
```
|
|
|
|
`SourceLine(src)` renders the source line the error points at from `src`,
|
|
followed by a caret line marking the column, for messages the reader sees
|
|
under the input.
|
|
|
|
### `type DecodeError struct{ Path Path; Err error }`
|
|
|
|
Wraps a decoding failure with the key path at which it happened. `Path` lists
|
|
one segment per level from the document root, the outermost key first: a key
|
|
contributes its name, an array element its bracketed index, so the path of the
|
|
`weight` field in the first item reads `["items", "[0]", "weight"]` and its
|
|
`String()` renders `items[0].weight`. Read the fields instead of parsing the
|
|
message:
|
|
|
|
```go
|
|
if de, ok := errors.AsType[*interpres.DecodeError](err); ok {
|
|
fmt.Println(de.Path.String(), de.Err)
|
|
}
|
|
```
|
|
|
|
### `type EncodeError struct{ Path Path; Err error }`
|
|
|
|
Wraps an encoding failure with the key path of the value that failed, the
|
|
same `Path` type the decode error carries, so `server.ports[2]` reads the
|
|
same on both sides. Read it with `errors.AsType` the same way.
|
|
|
|
### `type Path []string`
|
|
|
|
The path both error wrappers carry, one segment per level from the document
|
|
root. `String()` renders the TOML notation: keys join with dots, an index
|
|
attaches to the previous segment in brackets, `items[0].weight`.
|
|
|
|
### Options
|
|
|
|
The decode and encode entries take variadic options, the shape
|
|
encoding/json/v2 uses for its own. Each option is a stateless function value
|
|
over the private settings of one call; they compose by listing in the call,
|
|
and there is no stateful Decoder or Encoder to share or guard.
|
|
|
|
Decode options:
|
|
|
|
| Option | Default | Effect |
|
|
|---|---|---|
|
|
| `RejectUnknownFields(v bool)` | off | a key with no matching struct field is an error |
|
|
| `NumbersAsLiterals(v bool)` | off | integers and floats decode into `Number`, which carries the literal; see [Numbers as literals](#numbers-as-literals) |
|
|
| `MaxNestingDepth(depth int)` | `10000` | bound how deeply arrays and inline tables may nest |
|
|
| `MaxInputSize(size int)` | no limit | bound the size of the document, in bytes |
|
|
| `LocalTimeLocation(loc)` | nil | the zone a local date-time is carried in when it decodes into a `time.Time` |
|
|
|
|
Encode options:
|
|
|
|
| Option | Default | Effect |
|
|
|---|---|---|
|
|
| `Layout(kind LayoutKind)` | `LayoutKindGrouped` | group entries as scalars, then sub-tables, then arrays of tables; `LayoutKindDeclaration` preserves declaration order |
|
|
| `OmitEmptyArrays(v bool)` | off | skip `key = []` for empty scalar arrays |
|
|
| `LiteralMultiline(threshold int)` | `0` | emit multi-line strings of at least `threshold` bytes as literal `'''...'''` |
|
|
| `InlineTables(threshold int)` | `0` | write a sub-table inline when its single-line form is at most `threshold` bytes |
|
|
| `EmitFieldComments(v bool)` | off | print the `comment=` tag option of a field above its line or header |
|
|
|
|
```go
|
|
out, err := interpres.MarshalContext(ctx, cfg,
|
|
interpres.Layout(interpres.LayoutKindDeclaration),
|
|
interpres.OmitEmptyArrays(true),
|
|
interpres.LiteralMultiline(80),
|
|
interpres.InlineTables(60))
|
|
```
|
|
|
|
The nesting limit protects the stack, because the parser is a recursive
|
|
descent: a deeper document is rejected with a `SyntaxError` naming the limit
|
|
rather than running the stack out. `Parse` and `ParseContext` carry that same
|
|
default but take no options. The size limit is off by default, because the
|
|
caller already holds the bytes and the size is therefore a policy, not a
|
|
protection the library can impose on its own.
|
|
|
|
### `type Document`, `type Table`, `type Entry`
|
|
|
|
See [Documents](#documents). A `Document` is what `Parse` returns, and
|
|
`Marshal` writes it back: the keys in written order, the comments in place,
|
|
the inline tables inline. `UnmarshalDocument(doc, v)` decodes it without
|
|
parsing again.
|
|
|
|
### `type Marshaler interface{ MarshalTOML() (any, error) }`
|
|
|
|
See [Custom encoding](#custom-encoding-marshaler).
|
|
|
|
### `type Unmarshaler interface{ UnmarshalTOML(data any) error }`
|
|
|
|
See [Custom decoding](#custom-decoding-unmarshaler). `UnmarshalerContext`
|
|
carries the decode's context through `UnmarshalTOMLContext(ctx, data)` and
|
|
wins when a type implements both.
|
|
|
|
### `type Number string`
|
|
|
|
The literal a number was written with, what `NumbersAsLiterals` decodes into and what
|
|
`Marshal` writes back as it is. See
|
|
[Numbers as literals](#numbers-as-literals).
|
|
|
|
### `type OrderedMap`
|
|
|
|
The string-keyed table that keeps its key order on both the encode and the
|
|
decode side. See [Ordered tables](#ordered-tables).
|
|
|
|
### Date-time wrappers
|
|
|
|
```go
|
|
type OffsetDateTime struct{ time.Time } // 1979-05-27T07:32:00Z
|
|
type LocalDateTime struct{ time.Time } // 1979-05-27T07:32:00
|
|
type LocalDate struct{ time.Time } // 1979-05-27
|
|
type LocalTime struct{ time.Time } // 07:32:00.999999
|
|
```
|
|
|
|
Each carries a `String()` method returning the TOML-canonical rendering: the
|
|
seconds appear only when the value carries them, and a fractional second drops
|
|
its trailing zeros, so `07:32:00` renders as `07:32` and a half second as
|
|
`00.5`. The types are produced by `Parse` and accepted by `Marshal`, which
|
|
writes them through `String()`.
|
|
|
|
## Errors
|
|
|
|
The entry points return:
|
|
|
|
- `*SyntaxError` for a malformed document, with the 1-based line; the nesting
|
|
limit reports through it as well
|
|
- `*DecodeError` for a decoding failure, with the key path in `Path`
|
|
- `*EncodeError` for an encoding failure, with the key path in `Path`
|
|
- a plain error for the rest: a non-pointer decode target, a cancelled
|
|
context, a key that is not valid UTF-8, an input over the size limit
|
|
|
|
Decode and encode failures carry the key path or element index in the typed
|
|
wrappers above, so `errors.Is` and `errors.AsType` see through them and the
|
|
path reads from a field instead of the message text.
|
|
|
|
## Notes
|
|
|
|
The exported surface is documented in godoc form in the source, and `go doc .`
|
|
run from the module root is the authority on signatures and types. This file
|
|
explains what the surface is for and how the parts fit together.
|