7 Commits
Author SHA1 Message Date
petrbalvin bccaf087c8 feat(cmd): add the encoder mode to the toml-test adapter
Test / test (push) Successful in 1m33s
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 11:38:19 +02:00
petrbalvin 0149a5b4d1 docs(encoder): correct the multi-line string claim
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 11:38:17 +02:00
petrbalvin 815141440e feat: honour TextMarshaler and TextUnmarshaler by default
Test / test (push) Successful in 1m35s
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 02:41:09 +02:00
petrbalvin 9023784da3 fix(decode): decode into a defined string or bool type
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 02:40:51 +02:00
petrbalvin 942c4b1489 docs(security): list the newest release as supported
Test / test (push) Successful in 1m33s
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 02:24:24 +02:00
petrbalvin 8f0eae6604 docs: drop the TOML 1.0 compatibility promise
Assisted-by: DeepSeek V4.1 Flash
2026-09-19 02:24:24 +02:00
petrbalvin 1c7329aeea build: move the module path to /v2
Test / test (push) Successful in 1m32s
Assisted-by: GLM 5.3 Flash
2026-09-19 00:14:39 +02:00
19 changed files with 1120 additions and 62 deletions
+5 -3
View File
@@ -105,6 +105,8 @@ jobs:
run: go build -o bin/interpres-decode ./cmd/interpres-decode run: go build -o bin/interpres-decode ./cmd/interpres-decode
- name: Compliance suite - name: Compliance suite
# interpres implements TOML 1.0 and 1.1; the mode is pinned so an upstream # interpres implements TOML 1.1, and the suite runs both directions: the decoder
# default change cannot silently move the corpus. # on the valid and invalid corpora, the encoder on the tagged JSON of the valid
run: bin/toml-test test -decoder=bin/interpres-decode -toml=1.1 # one. The mode is pinned so an upstream default change cannot silently move the
# corpus.
run: bin/toml-test test -decoder=bin/interpres-decode -encoder='bin/interpres-decode -encode' -toml=1.1
+34 -1
View File
@@ -9,7 +9,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added ### Added
- - `encoding.TextMarshaler` and `encoding.TextUnmarshaler` are honoured by
default, with no option to switch them off. A type that implements them is
encoded as a TOML string and decoded from one: `net.IP` becomes
`"192.0.2.1"`, and a user type with `MarshalText` or `UnmarshalText` follows.
`MarshalTOML` and `UnmarshalTOML` still win over the text methods, and the
four date-time types keep their bare timestamp form instead of becoming a
quoted string. A struct type that implements the interface now encodes as a
string where it was a table before, which is the breaking part of the change.
- `time.Duration` is encoded in its canonical Go form as a TOML string,
`1h30m0s`, because TOML has no duration type; the decoder reads that string
back and still accepts a bare integer as the nanosecond count.
- `interpres-decode -encode`, the adapter's other direction: it reads the
toml-test tagged JSON from stdin and writes the TOML document it describes.
The compliance suite now runs the encoder as well as the decoder, 214
encoder cases against the tagged JSON of the valid corpus.
### Changed
- TOML 1.1 is the acceptance contract, and TOML 1.0 is not. The compliance
suite runs the 1.1 corpus alone, and the promise that every 1.0 document
parses exactly as before is withdrawn. Nothing that parses today stops
parsing: the 1.0 valid corpus still passes in full. The documents whose
verdict changes are the ones 1.1 relaxed, such as the `\xHH` escape
sequences 1.0 rejected.
- The module path carries the /v2 suffix the Go toolchain requires of
every major version 2 module: imports change to
`sourcedock.dev/petrbalvin/interpres/v2`.
### Fixed
- Decoding into a defined type whose underlying kind is string or bool, such
as `type Name string`, panicked instead of storing the value, because a
value of the predeclared type is not assignable to a defined type and the
decoder assigned it without a conversion.
## [1.1.0] - 2026-09-18 ## [1.1.0] - 2026-09-18
+2 -2
View File
@@ -45,8 +45,8 @@ just test
formatting pass are three commits, never one. formatting pass are three commits, never one.
4. Record every user-visible change in `CHANGELOG.md` under `## [development]`. 4. Record every user-visible change in `CHANGELOG.md` under `## [development]`.
5. Add or update tests. Coverage stays at 80 percent or more; it is a hard 5. Add or update tests. Coverage stays at 80 percent or more; it is a hard
gate. Parser and decoder changes must also keep the toml-test suite at zero gate. Parser, decoder and encoder changes must also keep both directions of
failures, checked with `just toml-test`. the toml-test suite at zero failures, checked with `just toml-test`.
6. Update the documentation when the public API, the configuration or the 6. Update the documentation when the public API, the configuration or the
behaviour changes; the documents move in the same commit as the behaviour behaviour changes; the documents move in the same commit as the behaviour
they describe. they describe.
+9 -7
View File
@@ -1,14 +1,14 @@
# interpres # interpres
A TOML 1.0 and 1.1 parser and encoder for Go, written with the standard A TOML 1.1 parser and encoder for Go, written with the standard library
library alone. `interpres` (Latin for *interpreter*) gives zero-dependency alone. `interpres` (Latin for *interpreter*) gives zero-dependency
programs an `encoding/json`-style API for reading and writing TOML, and passes programs an `encoding/json`-style API for reading and writing TOML, and passes
the entire official [toml-test](https://github.com/toml-lang/toml-test) suite: the entire official [toml-test](https://github.com/toml-lang/toml-test) suite:
214 valid and 467 invalid cases, zero failures. 214 valid, 467 invalid and 214 encoder cases, zero failures.
## Features ## Features
- **Full TOML 1.0 and 1.1**: bare, quoted and dotted keys; tables and arrays of - **Full TOML 1.1**: bare, quoted and dotted keys; tables and arrays of
tables; basic and literal strings including multiline, with the 1.1 `\e` and tables; basic and literal strings including multiline, with the 1.1 `\e` and
`\xHH` escapes; integers in the four radixes with `_` separators; floats with `\xHH` escapes; integers in the four radixes with `_` separators; floats with
exponents, `inf` and `nan`; booleans; the four date-time kinds, seconds exponents, `inf` and `nan`; booleans; the four date-time kinds, seconds
@@ -18,7 +18,9 @@ the entire official [toml-test](https://github.com/toml-lang/toml-test) suite:
- **Strict decoding**: `NewDecoder().DisallowUnknownFields()` rejects keys that - **Strict decoding**: `NewDecoder().DisallowUnknownFields()` rejects keys that
match no destination field, at every struct depth. match no destination field, at every struct depth.
- **Custom types**: `Marshaler` and `Unmarshaler` let a type control its own - **Custom types**: `Marshaler` and `Unmarshaler` let a type control its own
TOML representation in both directions. TOML representation in both directions, and `encoding.TextMarshaler` and
`TextUnmarshaler` are honoured by default, so `net.IP`, `time.Duration` and
user types with text methods need no configuration.
- **Cancellation**: every entry point has a `*Context` sibling that honours a - **Cancellation**: every entry point has a `*Context` sibling that honours a
`context.Context`. `context.Context`.
- **Configurable emission**: `Encoder` options for declaration-order output, - **Configurable emission**: `Encoder` options for declaration-order output,
@@ -29,7 +31,7 @@ the entire official [toml-test](https://github.com/toml-lang/toml-test) suite:
As a library: As a library:
```sh ```sh
go get sourcedock.dev/petrbalvin/interpres go get sourcedock.dev/petrbalvin/interpres/v2
``` ```
Requires Go 1.27.1 or newer. The module imports only the standard library. Requires Go 1.27.1 or newer. The module imports only the standard library.
@@ -161,7 +163,7 @@ See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for the full workflow, and
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): components and data flow - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): components and data flow
- [docs/API.md](docs/API.md): the API reference, decoding and encoding rules - [docs/API.md](docs/API.md): the API reference, decoding and encoding rules
- [docs/CLI.md](docs/CLI.md): the interpres-decode toml-test adapter and validator - [docs/CLI.md](docs/CLI.md): the interpres-decode adapter and validator
## Licence ## Licence
+1 -1
View File
@@ -7,7 +7,7 @@ releases do not receive them.
| Version | Supported | | Version | Supported |
|---|---| |---|---|
| 1.0.0 | yes | | 1.1.0 | yes |
| older releases | no | | older releases | no |
## Reporting a vulnerability ## Reporting a vulnerability
+196 -5
View File
@@ -4,14 +4,16 @@
// Command interpres-decode is the toml-test harness adapter and a TOML // Command interpres-decode is the toml-test harness adapter and a TOML
// validator. Without flags it reads a TOML document from standard input and // validator. Without flags it reads a TOML document from standard input and
// writes the toml-test "tagged JSON" representation to standard output. With // writes the toml-test "tagged JSON" representation to standard output. With
// -validate it checks the named documents, or standard input when none are // -encode it is the reverse: it reads tagged JSON and writes the TOML document
// named, and exits non-zero on the first invalid one: // it describes. With -validate it checks the named documents, or standard
// input when none are named, and exits non-zero on the first invalid one:
// //
// interpres-decode -validate config.toml // interpres-decode -validate config.toml
// interpres-decode -encode < case.json
// //
// Run the official suite against the adapter with: // Run the official suite in both directions against the adapter with:
// //
// toml-test ./interpres-decode // toml-test test -decoder=./interpres-decode -encoder='./interpres-decode -encode'
package main package main
import ( import (
@@ -25,7 +27,7 @@ import (
"strconv" "strconv"
"time" "time"
"sourcedock.dev/petrbalvin/interpres" "sourcedock.dev/petrbalvin/interpres/v2"
) )
func main() { func main() {
@@ -39,12 +41,17 @@ func Run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("interpres-decode", flag.ContinueOnError) fs := flag.NewFlagSet("interpres-decode", flag.ContinueOnError)
fs.SetOutput(stderr) fs.SetOutput(stderr)
validate := fs.Bool("validate", false, "validate the documents instead of emitting tagged JSON") validate := fs.Bool("validate", false, "validate the documents instead of emitting tagged JSON")
encode := fs.Bool("encode", false, "read tagged JSON from stdin and write TOML instead")
if err := fs.Parse(args); err != nil { if err := fs.Parse(args); err != nil {
if errors.Is(err, flag.ErrHelp) { if errors.Is(err, flag.ErrHelp) {
return 0 return 0
} }
return 2 return 2
} }
if *validate && *encode {
fmt.Fprintln(stderr, "interpres-decode: -validate and -encode cannot be combined")
return 2
}
if *validate { if *validate {
return validatePaths(fs.Args(), stdin, stderr) return validatePaths(fs.Args(), stdin, stderr)
} }
@@ -52,6 +59,9 @@ func Run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
fmt.Fprintln(stderr, "interpres-decode: the adapter mode takes no arguments; name files with -validate") fmt.Fprintln(stderr, "interpres-decode: the adapter mode takes no arguments; name files with -validate")
return 2 return 2
} }
if *encode {
return encodeJSON(stdin, stdout, stderr)
}
data, err := io.ReadAll(stdin) data, err := io.ReadAll(stdin)
if err != nil { if err != nil {
fmt.Fprintln(stderr, "read stdin:", err) fmt.Fprintln(stderr, "read stdin:", err)
@@ -109,6 +119,187 @@ func validatePaths(paths []string, stdin io.Reader, stderr io.Writer) int {
return 0 return 0
} }
// encodeJSON reads a toml-test tagged JSON description from standard input and
// writes the TOML document it describes to standard output.
func encodeJSON(stdin io.Reader, stdout, stderr io.Writer) int {
data, err := io.ReadAll(stdin)
if err != nil {
fmt.Fprintln(stderr, "read stdin:", err)
return 2
}
var desc any
if err := json.Unmarshal(data, &desc); err != nil {
fmt.Fprintln(stderr, "decode JSON:", err)
return 2
}
tree, err := untag(desc)
if err != nil {
fmt.Fprintln(stderr, err)
return 2
}
doc, ok := tree.(map[string]any)
if !ok {
fmt.Fprintln(stderr, "interpres-decode: the description must be a JSON object at the top level")
return 2
}
out, err := interpres.Marshal(doc)
if err != nil {
fmt.Fprintln(stderr, err)
return 2
}
if _, err := stdout.Write(out); err != nil {
fmt.Fprintln(stderr, "write stdout:", err)
return 2
}
return 0
}
// untag converts a toml-test JSON description into the value tree Marshal
// expects: a JSON object becomes a map[string]any, a JSON array becomes a
// []any, and an object carrying exactly the keys "type" and "value" becomes
// the Go value for that TOML type.
func untag(v any) (any, error) {
switch x := v.(type) {
case map[string]any:
if typ, val, ok := taggedValue(x); ok {
return decodeTagged(typ, val)
}
out := make(map[string]any, len(x))
for k, e := range x {
u, err := untag(e)
if err != nil {
return nil, fmt.Errorf("%s: %w", k, err)
}
out[k] = u
}
return out, nil
case []any:
out := make([]any, len(x))
for i, e := range x {
u, err := untag(e)
if err != nil {
return nil, fmt.Errorf("[%d]: %w", i, err)
}
out[i] = u
}
return asTables(out), nil
default:
return nil, fmt.Errorf("unsupported JSON value %T", v)
}
}
// asTables returns the elements as a []map[string]any when there is at least
// one and every element is a table, the shape the encoder renders as an array
// of tables. The tagged JSON cannot tell an array of tables from a value array
// of inline tables, and both parse back to the same value, so the header form
// is chosen because it is the one the encoder otherwise never exercises. An
// empty array stays a []any, because TOML has no empty array of tables.
func asTables(items []any) any {
if len(items) == 0 {
return items
}
tbls := make([]map[string]any, len(items))
for i, e := range items {
tbl, ok := e.(map[string]any)
if !ok {
return items
}
tbls[i] = tbl
}
return tbls
}
// taggedValue reports whether m is a toml-test value object: a JSON object of
// exactly the two string keys "type" and "value", carrying a type this adapter
// knows. Any other object is a table.
func taggedValue(m map[string]any) (typ, val string, ok bool) {
if len(m) != 2 {
return "", "", false
}
ts, ok := m["type"].(string)
if !ok || !knownType(ts) {
return "", "", false
}
vs, ok := m["value"].(string)
if !ok {
return "", "", false
}
return ts, vs, true
}
func knownType(typ string) bool {
switch typ {
case "string", "integer", "float", "bool",
"datetime", "datetime-local", "date-local", "time-local":
return true
}
return false
}
// decodeTagged returns the Go value for one tagged JSON value. Every type but
// string is parsed by the library itself, so the adapter and the library agree
// on what an integer, a float or a date-time is.
func decodeTagged(typ, val string) (any, error) {
if typ == "string" {
return val, nil
}
v, err := parseAtom(val)
if err != nil {
return nil, fmt.Errorf("%s %q: %w", typ, val, err)
}
// A float with no fractional part and no exponent is described by a bare
// integer literal, so here the tag decides and not the literal.
if n, ok := v.(int64); ok && typ == "float" {
return float64(n), nil
}
if !typeMatches(typ, v) {
return nil, fmt.Errorf("%s %q parsed as %T", typ, val, v)
}
return v, nil
}
// parseAtom parses one bare TOML value, by handing `v = <val>` to the library's
// parser and requiring the result to hold exactly that one statement, so a
// value carrying a newline or a comment cannot smuggle a second one in.
func parseAtom(val string) (any, error) {
tree, err := interpres.Parse([]byte("v = " + val + "\n"))
if err != nil {
return nil, err
}
if len(tree) != 1 {
return nil, errors.New("not a single bare value")
}
return tree["v"], nil
}
// typeMatches reports whether v is the Go value the tagged type names.
func typeMatches(typ string, v any) bool {
switch typ {
case "integer":
_, ok := v.(int64)
return ok
case "float":
_, ok := v.(float64)
return ok
case "bool":
_, ok := v.(bool)
return ok
case "datetime":
_, ok := v.(time.Time)
return ok
case "datetime-local":
_, ok := v.(interpres.LocalDateTime)
return ok
case "date-local":
_, ok := v.(interpres.LocalDate)
return ok
case "time-local":
_, ok := v.(interpres.LocalTime)
return ok
}
return false
}
// tag converts an interpres value into its toml-test tagged-JSON form. Tables // tag converts an interpres value into its toml-test tagged-JSON form. Tables
// become JSON objects and arrays become JSON arrays; scalars are wrapped in a // become JSON objects and arrays become JSON arrays; scalars are wrapped in a
// {"type", "value"} object. An error is returned for value types the encoder // {"type", "value"} object. An error is returned for value types the encoder
+164 -1
View File
@@ -8,11 +8,12 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"os" "os"
"reflect"
"strings" "strings"
"testing" "testing"
"time" "time"
"sourcedock.dev/petrbalvin/interpres" "sourcedock.dev/petrbalvin/interpres/v2"
) )
func TestRunParsesValidTOML(t *testing.T) { func TestRunParsesValidTOML(t *testing.T) {
@@ -283,3 +284,165 @@ func TestUnknownFlagReturnsTwo(t *testing.T) {
t.Fatalf("Run returned %d, want 2; stderr = %q", code, stderr.String()) t.Fatalf("Run returned %d, want 2; stderr = %q", code, stderr.String())
} }
} }
// --- encoder mode ----------------------------------------------------------
func TestRunEncoderScalars(t *testing.T) {
in := `{
"s": {"type": "string", "value": "quote \" and backslash \\"},
"nl": {"type": "string", "value": "line1\nline2"},
"i": {"type": "integer", "value": "-9223372036854775808"},
"g": {"type": "float", "value": "1.5"},
"f": {"type": "float", "value": "inf"},
"b": {"type": "bool", "value": "false"},
"dt": {"type": "datetime", "value": "1979-05-27T07:32:00-07:00"},
"ldt": {"type": "datetime-local", "value": "1979-05-27T07:32:00"},
"ld": {"type": "date-local", "value": "1979-05-27"},
"lt": {"type": "time-local", "value": "07:32:00.999"}
}
`
var stdout, stderr bytes.Buffer
code := Run([]string{"-encode"}, strings.NewReader(in), &stdout, &stderr)
if code != 0 {
t.Fatalf("Run returned %d, stderr = %q", code, stderr.String())
}
want := "b = false\n" +
"dt = 1979-05-27T07:32:00-07:00\n" +
"f = inf\n" +
"g = 1.5\n" +
"i = -9223372036854775808\n" +
"ld = 1979-05-27\n" +
"ldt = 1979-05-27T07:32:00\n" +
"lt = 07:32:00.999000000\n" +
"nl = \"line1\\nline2\"\n" +
"s = \"quote \\\" and backslash \\\\\"\n"
if stdout.String() != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", stdout.String(), want)
}
}
func TestRunEncoderNested(t *testing.T) {
in := `{
"tbl": {"x": {"type": "bool", "value": "true"},
"sub": {"y": {"type": "integer", "value": "1"}}},
"items": [{"n": {"type": "string", "value": "a"}},
{"n": {"type": "string", "value": "b"}}],
"list": [{"type": "integer", "value": "1"}, {"type": "string", "value": "two"}],
"emptyTbl": {},
"emptyArr": []
}
`
var stdout, stderr bytes.Buffer
code := Run([]string{"-encode"}, strings.NewReader(in), &stdout, &stderr)
if code != 0 {
t.Fatalf("Run returned %d, stderr = %q", code, stderr.String())
}
want := "emptyArr = []\n" +
"list = [1, \"two\"]\n" +
"\n[emptyTbl]\n" +
"\n[tbl]\nx = true\n" +
"\n[tbl.sub]\ny = 1\n" +
"\n[[items]]\nn = \"a\"\n" +
"\n[[items]]\nn = \"b\"\n"
if stdout.String() != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", stdout.String(), want)
}
}
func TestRunEncoderFloatTagDecides(t *testing.T) {
// A float with no fraction is described by a bare integer literal, so the
// tag decides the type; the output must stay a float.
var stdout, stderr bytes.Buffer
in := `{"whole": {"type": "float", "value": "1"}, "exp": {"type": "float", "value": "5e+22"}}`
code := Run([]string{"-encode"}, strings.NewReader(in), &stdout, &stderr)
if code != 0 {
t.Fatalf("Run returned %d, stderr = %q", code, stderr.String())
}
if want := "exp = 5e+22\nwhole = 1.0\n"; stdout.String() != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", stdout.String(), want)
}
}
func TestRunEncoderRejectsBadInput(t *testing.T) {
cases := []struct {
name string
in string
want string
}{
{"not-json", "not json", "decode JSON"},
{"top-level-array", `[{"type": "integer", "value": "1"}]`, "must be a JSON object"},
{"untagged-scalar", `{"x": 1}`, "unsupported JSON value"},
{"literal-mismatch", `{"x": {"type": "integer", "value": "1.5"}}`, "parsed as float64"},
{"offset-for-local", `{"x": {"type": "datetime-local", "value": "1979-05-27T07:32:00Z"}}`, "parsed as time.Time"},
{"bad-literal", `{"x": {"type": "date-local", "value": "nope"}}`, "date-local"},
{"smuggled-statement", `{"x": {"type": "integer", "value": "1\nx = 2"}}`, "not a single bare value"},
}
for _, c := range cases {
var stdout, stderr bytes.Buffer
code := Run([]string{"-encode"}, strings.NewReader(c.in), &stdout, &stderr)
if code != 2 {
t.Errorf("%s: Run returned %d, want 2; stderr = %q", c.name, code, stderr.String())
continue
}
if !strings.Contains(stderr.String(), c.want) {
t.Errorf("%s: stderr = %q, want it to mention %q", c.name, stderr.String(), c.want)
}
if stdout.Len() != 0 {
t.Errorf("%s: stdout should be empty, got %q", c.name, stdout.String())
}
}
}
func TestRunEncoderFlagConflicts(t *testing.T) {
var stdout, stderr bytes.Buffer
if code := Run([]string{"-encode", "-validate"}, strings.NewReader(""), &stdout, &stderr); code != 2 {
t.Errorf("Run returned %d, want 2 for the two modes together", code)
}
if !strings.Contains(stderr.String(), "cannot be combined") {
t.Errorf("stderr = %q, want it to explain the conflict", stderr.String())
}
stdout.Reset()
stderr.Reset()
if code := Run([]string{"-encode", "file.json"}, strings.NewReader(""), &stdout, &stderr); code != 2 {
t.Errorf("Run returned %d, want 2 for an argument", code)
}
}
func TestEncodeAfterDecodeRoundTrip(t *testing.T) {
doc := `title = "x"
flt = 1.5
whole = 7.0
big = 9223372036854775807
when = 1979-05-27T07:32:00-07:00
day = 1979-05-27
clock = 07:32:00.999
list = [1, "two"]
multi = "a\nb"
[tbl]
x = true
[[items]]
n = "a"
`
var tagged, stderr bytes.Buffer
if code := Run(nil, strings.NewReader(doc), &tagged, &stderr); code != 0 {
t.Fatalf("decode returned %d, stderr = %q", code, stderr.String())
}
var out bytes.Buffer
if code := Run([]string{"-encode"}, bytes.NewReader(tagged.Bytes()), &out, &stderr); code != 0 {
t.Fatalf("encode returned %d, stderr = %q", code, stderr.String())
}
want, err := interpres.Parse([]byte(doc))
if err != nil {
t.Fatalf("parse of the original: %v", err)
}
got, err := interpres.Parse(out.Bytes())
if err != nil {
t.Fatalf("parse of the encoder output (%q): %v", out.String(), err)
}
if !reflect.DeepEqual(want, got) {
t.Errorf("round trip changed the document:\noriginal: %#v\nencoded: %#v\noutput: %q", want, got, out.String())
}
}
+54 -1
View File
@@ -4,6 +4,7 @@
package interpres package interpres
import ( import (
"encoding"
"fmt" "fmt"
"reflect" "reflect"
"slices" "slices"
@@ -63,6 +64,19 @@ func (d *decoder) assign(data any, dst reflect.Value) error {
} }
} }
// A TOML string fills a destination that implements
// encoding.TextUnmarshaler, the rule encoding/json follows. Every other
// value kind keeps its own rule, so an integer still reaches a numeric
// destination.
if s, isString := data.(string); isString {
if tu, ok := textUnmarshalerOf(dst); ok {
if err := tu.UnmarshalText([]byte(s)); err != nil {
return fmt.Errorf("unmarshal text: %w", err)
}
return nil
}
}
switch v := data.(type) { switch v := data.(type) {
case map[string]any: case map[string]any:
return d.assignTable(v, dst) return d.assignTable(v, dst)
@@ -71,6 +85,9 @@ func (d *decoder) assign(data any, dst reflect.Value) error {
case []any: case []any:
return d.assignSlice(v, dst) return d.assignSlice(v, dst)
case string: case string:
if dst.Type() == durationType {
return setDuration(dst, v)
}
return setBasic(dst, reflect.ValueOf(v), "string") return setBasic(dst, reflect.ValueOf(v), "string")
case bool: case bool:
return setBasic(dst, reflect.ValueOf(v), "bool") return setBasic(dst, reflect.ValueOf(v), "bool")
@@ -94,6 +111,26 @@ func (d *decoder) assign(data any, dst reflect.Value) error {
} }
} }
// textUnmarshalerOf finds the encoding.TextUnmarshaler for dst: on the value
// itself, or on its address, so a pointer-receiver UnmarshalText is invoked on
// an addressable struct field. The TOML date-time types are excluded, because
// they carry time.Time's UnmarshalText through an embedded field while their
// only accepted form is a bare timestamp.
func textUnmarshalerOf(dst reflect.Value) (encoding.TextUnmarshaler, bool) {
if !dst.CanInterface() || isDateTimeType(dst.Type()) {
return nil, false
}
if u, ok := dst.Interface().(encoding.TextUnmarshaler); ok {
return u, true
}
if dst.CanAddr() {
if u, ok := dst.Addr().Interface().(encoding.TextUnmarshaler); ok {
return u, true
}
}
return nil, false
}
func (d *decoder) assignTable(tbl map[string]any, dst reflect.Value) error { func (d *decoder) assignTable(tbl map[string]any, dst reflect.Value) error {
switch dst.Kind() { switch dst.Kind() {
case reflect.Struct: case reflect.Struct:
@@ -202,7 +239,23 @@ func setBasic(dst, val reflect.Value, kind string) error {
if dst.Kind() != val.Kind() { if dst.Kind() != val.Kind() {
return fmt.Errorf("interpres: cannot assign %s to %s", kind, dst.Type()) return fmt.Errorf("interpres: cannot assign %s to %s", kind, dst.Type())
} }
dst.Set(val) // Convert rather than assign: a value of the predeclared type is not
// assignable to a defined type of the same kind, so a plain Set panics on
// a destination such as `type Name string`.
dst.Set(val.Convert(dst.Type()))
return nil
}
// setDuration reads a duration literal into a time.Duration destination. TOML
// has no duration type, so the encoder writes the canonical Go form and the
// decoder reads that back; a bare integer stays the nanosecond count it has
// always been, and reaches the destination through setInt.
func setDuration(dst reflect.Value, s string) error {
d, err := time.ParseDuration(s)
if err != nil {
return fmt.Errorf("interpres: invalid duration %q", s)
}
dst.SetInt(int64(d))
return nil return nil
} }
+226
View File
@@ -8,9 +8,11 @@ import (
"errors" "errors"
"fmt" "fmt"
"math" "math"
"net"
"slices" "slices"
"strings" "strings"
"testing" "testing"
"time"
) )
func TestSyntaxErrorMessage(t *testing.T) { func TestSyntaxErrorMessage(t *testing.T) {
@@ -725,3 +727,227 @@ func TestDecodeErrorOnMapDestination(t *testing.T) {
t.Fatalf("Path = %v", de.Path) t.Fatalf("Path = %v", de.Path)
} }
} }
func TestUnmarshalIntoDefinedScalarTypes(t *testing.T) {
// A defined type whose underlying kind is string or bool takes the value.
// A bare reflect Set panics on such a type, because a string is not
// assignable to a defined string type without a conversion.
type Name string
type Flag bool
type Cfg struct {
N Name `toml:"n"`
F Flag `toml:"f"`
}
var cfg Cfg
if err := Unmarshal([]byte("n = \"x\"\nf = true\n"), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.N != "x" {
t.Errorf("N = %q, want \"x\"", cfg.N)
}
if !cfg.F {
t.Error("F = false, want true")
}
}
// --- encoding.TextUnmarshaler and time.Duration ----------------------------
// textReceiver implements encoding.TextUnmarshaler on the pointer receiver.
type textReceiver struct{ Text string }
func (t *textReceiver) UnmarshalText(text []byte) error {
t.Text = "got:" + string(text)
return nil
}
// upperText is a defined string type whose UnmarshalText transforms the
// content, so a plain string assignment would leave the wrong value behind.
type upperText string
func (u *upperText) UnmarshalText(text []byte) error {
*u = upperText(strings.ToUpper(string(text)))
return nil
}
// failingTextUnmarshaler fails the decode from UnmarshalText.
type failingTextUnmarshaler struct{}
func (f *failingTextUnmarshaler) UnmarshalText(_ []byte) error { return errors.New("text boom") }
// textAndTOMLReceiver implements both decode interfaces; the TOML method wins.
type textAndTOMLReceiver struct{ From string }
func (t *textAndTOMLReceiver) UnmarshalTOML(any) error { t.From = "toml"; return nil }
func (t *textAndTOMLReceiver) UnmarshalText([]byte) error { t.From = "text"; return nil }
func TestTextUnmarshalerByPointer(t *testing.T) {
type Cfg struct {
R textReceiver `toml:"r"`
}
var cfg Cfg
if err := Unmarshal([]byte(`r = "hello"`), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.R.Text != "got:hello" {
t.Errorf("Text = %q, want \"got:hello\"", cfg.R.Text)
}
}
func TestTextUnmarshalerWinsOverKindAssignment(t *testing.T) {
type Cfg struct {
U upperText `toml:"u"`
}
var cfg Cfg
if err := Unmarshal([]byte(`u = "abc"`), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.U != "ABC" {
t.Errorf("U = %q, want \"ABC\"", cfg.U)
}
}
func TestTextUnmarshalerForNetIP(t *testing.T) {
type Cfg struct {
V4 net.IP `toml:"v4"`
V6 net.IP `toml:"v6"`
IPs []net.IP `toml:"ips"`
}
in := "v4 = \"192.0.2.1\"\nv6 = \"2001:db8::68\"\nips = [\"198.51.100.7\", \"203.0.113.9\"]\n"
var cfg Cfg
if err := Unmarshal([]byte(in), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if got := cfg.V4.String(); got != "192.0.2.1" {
t.Errorf("V4 = %q, want \"192.0.2.1\"", got)
}
if got := cfg.V6.String(); got != "2001:db8::68" {
t.Errorf("V6 = %q, want \"2001:db8::68\"", got)
}
if len(cfg.IPs) != 2 || cfg.IPs[0].String() != "198.51.100.7" || cfg.IPs[1].String() != "203.0.113.9" {
t.Errorf("IPs = %v, want two addresses", cfg.IPs)
}
}
func TestTextUnmarshalerSeesStringsOnly(t *testing.T) {
// An integer keeps its own rule: the text method is not consulted, and the
// value does not reach the receiver.
type Cfg struct {
R textReceiver `toml:"r"`
}
var cfg Cfg
err := Unmarshal([]byte("r = 1\n"), &cfg)
if err == nil {
t.Fatal("expected an integer to be rejected for a text receiver")
}
if cfg.R.Text != "" {
t.Errorf("Text = %q, want it untouched", cfg.R.Text)
}
}
func TestUnmarshalTOMLWinsOverTextUnmarshaler(t *testing.T) {
type Cfg struct {
B textAndTOMLReceiver `toml:"b"`
}
var cfg Cfg
if err := Unmarshal([]byte(`b = "x"`), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.B.From != "toml" {
t.Errorf("From = %q, want \"toml\"", cfg.B.From)
}
}
func TestTextUnmarshalerErrorCarriesPath(t *testing.T) {
type Inner struct {
F failingTextUnmarshaler `toml:"f"`
}
type Cfg struct {
Inner Inner `toml:"inner"`
}
var cfg Cfg
err := Unmarshal([]byte("[inner]\nf = \"x\"\n"), &cfg)
if err == nil {
t.Fatal("expected an error from UnmarshalText")
}
if !strings.Contains(err.Error(), "unmarshal text: text boom") {
t.Errorf("err = %v, want the text error wrapped", err)
}
de, ok := errors.AsType[*DecodeError](err)
if !ok {
t.Fatalf("expected a *DecodeError, got %T: %v", err, err)
}
if !slices.Equal(de.Path, []string{"inner", "f"}) {
t.Fatalf("Path = %v, want [inner f]", de.Path)
}
}
func TestTextUnmarshalerReportsBadText(t *testing.T) {
var cfg struct {
IP net.IP `toml:"ip"`
}
err := Unmarshal([]byte(`ip = "not-an-ip"`), &cfg)
if err == nil {
t.Fatal("expected an error for a malformed address")
}
if !strings.Contains(err.Error(), "unmarshal text:") {
t.Errorf("err = %v, want it wrapped as a text error", err)
}
}
func TestUnmarshalDurations(t *testing.T) {
type Cfg struct {
FromText time.Duration `toml:"from_text"`
FromInt time.Duration `toml:"from_int"`
Fraction time.Duration `toml:"fraction"`
}
in := "from_text = \"1h30m\"\nfrom_int = 5400000000000\nfraction = \"1.5s\"\n"
var cfg Cfg
if err := Unmarshal([]byte(in), &cfg); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if cfg.FromText != 90*time.Minute {
t.Errorf("FromText = %v, want %v", cfg.FromText, 90*time.Minute)
}
if cfg.FromInt != 90*time.Minute {
t.Errorf("FromInt = %v, want %v", cfg.FromInt, 90*time.Minute)
}
if cfg.Fraction != 1500*time.Millisecond {
t.Errorf("Fraction = %v, want %v", cfg.Fraction, 1500*time.Millisecond)
}
}
func TestUnmarshalDurationRejectsMalformedText(t *testing.T) {
var cfg struct {
D time.Duration `toml:"d"`
}
err := Unmarshal([]byte("d = \"90\"\n"), &cfg)
if err == nil {
t.Fatal("expected an error for a duration without a unit")
}
if !strings.Contains(err.Error(), "invalid duration") {
t.Errorf("err = %v, want an invalid-duration message", err)
}
}
func TestQuotedStringNeverBecomesDateTime(t *testing.T) {
// The date-time types take a bare timestamp only, so the text path is
// excluded for them and a quoted string stays a string.
var stamp struct {
S time.Time `toml:"s"`
}
err := Unmarshal([]byte("s = \"2026-06-26T10:00:00Z\"\n"), &stamp)
if err == nil {
t.Fatal("expected a quoted string to be rejected for time.Time")
}
if !strings.Contains(err.Error(), "cannot assign string") {
t.Errorf("err = %v, want a cannot-assign message", err)
}
var day struct {
D LocalDate `toml:"d"`
}
if err := Unmarshal([]byte("d = \"1979-05-27\"\n"), &day); err == nil {
t.Fatal("expected a quoted string to be rejected for LocalDate")
}
}
+64 -10
View File
@@ -1,16 +1,15 @@
# API # API
The library exports the surface below from the `sourcedock.dev/petrbalvin/interpres` The library exports the surface below from the `sourcedock.dev/petrbalvin/interpres/v2`
package. The snippets assume: package. The snippets assume:
```go ```go
import "sourcedock.dev/petrbalvin/interpres" import "sourcedock.dev/petrbalvin/interpres/v2"
``` ```
The parser accepts TOML 1.0 documents plus the TOML 1.1 extensions: date-times The parser implements TOML 1.1: date-times and times without seconds, the
and times without seconds, the `\e` and `\xHH` escape sequences, and `\e` and `\xHH` escape sequences, and multi-line inline tables with comments
multi-line inline tables with comments and trailing commas. The encoder emits and trailing commas. The encoder emits TOML 1.1.
TOML 1.0, which is valid under both versions.
## Functions ## Functions
@@ -148,6 +147,9 @@ zero date for a local time). Every kind may omit the seconds as of TOML 1.1
(`07:32`, `1979-05-27T07:32`); such a value carries a zero second, and the (`07:32`, `1979-05-27T07:32`); such a value carries a zero second, and the
canonical rendering writes full seconds. There is no implicit conversion canonical rendering writes full seconds. There is no implicit conversion
between the offset and local kinds; assigning one to the other is an error. between the offset and local kinds; assigning one to the other is an error.
The four types take a bare timestamp and never a quoted string, so a document
that writes a date-time with quotes does not decode into them, and neither
`encoding.TextUnmarshaler` nor the embedded `time.Time` changes that.
### Arrays of tables ### Arrays of tables
@@ -178,6 +180,38 @@ automatically, and a nil pointer destination is allocated first. An error
returned from `UnmarshalTOML` halts the decode and propagates wrapped with the returned from `UnmarshalTOML` halts the decode and propagates wrapped with the
key path, for example `addr: unmarshal: not a string`. key path, for example `addr: unmarshal: not a string`.
### Custom decoding: `encoding.TextUnmarshaler`
A destination type that implements `encoding.TextUnmarshaler` receives a TOML
string as its text content, the rule `encoding/json` follows:
```go
func (ip *IP) UnmarshalText(text []byte) error
```
The decoder looks for the method on the destination and on its address, so a
pointer-receiver `UnmarshalText` is invoked on an addressable struct field, and
the elements of a slice destination are reached the same way. The text path
applies to TOML strings only: every other value kind keeps its own rule, so
`r = 1` does not reach a receiver that expects text. An error from
`UnmarshalText` halts the decode and propagates with the key path and the
prefix `unmarshal text:`, for example `addr: unmarshal text: not an address`.
[`UnmarshalTOML`](#custom-decoding-unmarshaler) wins over `UnmarshalText` when
a type implements both, and the four [date-time
types](#date-time-values) are excluded: a quoted string stays a string and
never becomes a `time.Time` or one of the local wrappers.
### Durations
TOML has no duration type, so `time.Duration` has a rule of its own. The
encoder writes the canonical Go form in a TOML string, `1h30m0s`, and the
decoder reads that string back with `time.ParseDuration`. A bare integer is
still the nanosecond count it has always been, so `from_int = 5400000000000`
and `from_text = "1h30m"` decode to the same duration. Text that
`time.ParseDuration` rejects, `d = "90"` among it, fails with
`interpres: invalid duration "90"`.
### Strict decoding ### Strict decoding
By default unknown keys are dropped silently. A `Decoder` built with By default unknown keys are dropped silently. A `Decoder` built with
@@ -330,6 +364,27 @@ func (p Port) MarshalTOML() (any, error) {
} }
``` ```
### Custom encoding: `encoding.TextMarshaler`
A type that implements `encoding.TextMarshaler` is encoded as a TOML string
holding the text the method returns, which is the rule `encoding/json` follows:
```go
func (ip IP) MarshalText() ([]byte, error)
```
The encoder looks for the method on the value and on its address, so a
pointer-receiver `MarshalText` is found on a struct field of an addressable
value (pass a pointer to `Marshal`) and always on a slice element. `net.IP`,
`netip.Addr` and user types follow this rule, and a struct that implements the
interface becomes a string rather than a table. `MarshalTOML` wins when a type
implements both, the four [date-time types](#date-time-values) keep their bare
timestamp form, and text that is not valid UTF-8 is an error rather than a
replacement character.
A duration carries no text method of its own; see [Durations](#durations) for
its rule.
### Arrays ### Arrays
An array whose every element is a table (`[]struct`, `[]map[string]V`, after An array whose every element is a table (`[]struct`, `[]map[string]V`, after
@@ -357,10 +412,9 @@ omitted, because TOML forbids an empty `[[a]]`. Other empty arrays emit as
### Long strings ### Long strings
By default every string is emitted as a basic `"..."` string with the escapes By default every string is emitted as a basic `"..."` string with the escapes
TOML requires, and a string containing a newline is emitted as an escaped TOML requires, a newline among them as `\n`. `UseLiteralMultiline(threshold)`
multi-line basic string. `UseLiteralMultiline(threshold)` switches strings that switches strings that contain a newline and are at least `threshold` bytes long
contain a newline and are at least `threshold` bytes long to the literal to the literal `'''...'''` form, which carries the newlines verbatim:
`'''...'''` form, which carries the newlines verbatim:
```go ```go
out, err := interpres.NewEncoder().UseLiteralMultiline(80).Marshal(cfg) out, err := interpres.NewEncoder().UseLiteralMultiline(80).Marshal(cfg)
+6 -6
View File
@@ -6,10 +6,10 @@ source tree; nothing is aspirational.
## Overview ## Overview
interpres is one public library package, one command, and one example. The interpres is one public library package, one command, and one example. The
library implements the whole of TOML 1.0 and 1.1, decoding and encoding, in the library implements the whole of TOML 1.1, decoding and encoding, in the
standard library alone; the command wraps the parser for the toml-test standard library alone; the command wraps the parser and the encoder for the
compliance harness, against which it stands at 214 valid and 467 invalid cases toml-test compliance harness, against which it stands at 214 valid, 467 invalid
with zero failures; the example demonstrates the API. and 214 encoder cases with zero failures; the example demonstrates the API.
```mermaid ```mermaid
flowchart TD flowchart TD
@@ -36,14 +36,14 @@ strict validation.
| Path | Responsibility | | Path | Responsibility |
|---|---| |---|---|
| `.` (package `interpres`) | The whole library. `interpres.go` declares the exported surface (`Parse`, `Unmarshal`, `Marshal`, the `*Context` variants, `Decoder`, `Encoder`, `Marshaler`, `Unmarshaler`, `SyntaxError`, the local date-time types); everything below it is unexported. | | `.` (package `interpres`) | The whole library. `interpres.go` declares the exported surface (`Parse`, `Unmarshal`, `Marshal`, the `*Context` variants, `Decoder`, `Encoder`, `Marshaler`, `Unmarshaler`, `SyntaxError`, the local date-time types); everything below it is unexported. |
| `cmd/interpres-decode` | The toml-test adapter. Reads TOML on stdin, writes tagged JSON on stdout. Owns no parsing logic. | | `cmd/interpres-decode` | The toml-test adapter, both directions. Reads TOML on stdin, writes tagged JSON on stdout; with `-encode` it reads tagged JSON and writes TOML. Owns no parsing logic and no emission logic. |
| `examples/basic` | A runnable tour of the API. Documentation in executable form, not part of the library. | | `examples/basic` | A runnable tour of the API. Documentation in executable form, not part of the library. |
Inside the library package, one file owns one concern: Inside the library package, one file owns one concern:
| File | Responsibility | | File | Responsibility |
|---|---| |---|---|
| `parser.go` | The recursive-descent parser. Produces the `map[string]any` tree and enforces the structural rules of TOML 1.0 and 1.1 (table redefinitions, dotted keys, arrays of tables, multi-line inline tables). Reports a 1-based line on failure. | | `parser.go` | The recursive-descent parser. Produces the `map[string]any` tree and enforces the structural rules of TOML 1.1 (table redefinitions, dotted keys, arrays of tables, multi-line inline tables). Reports a 1-based line on failure. |
| `number.go` | Strict numeric tokens: integers in the four radixes with `_` separators, and floats including `inf` and `nan`. Rejects leading zeros, misplaced underscores and malformed fractions. | | `number.go` | Strict numeric tokens: integers in the four radixes with `_` separators, and floats including `inf` and `nan`. Rejects leading zeros, misplaced underscores and malformed fractions. |
| `datetime.go` | The three local date-time wrapper types and `parseDateTime`, which classifies a token into the four date-time kinds under the strict TOML grammar. | | `datetime.go` | The three local date-time wrapper types and `parseDateTime`, which classifies a token into the four date-time kinds under the strict TOML grammar. |
| `decode.go` | Maps the parsed tree onto Go values by reflection: struct fields, maps, slices, scalar conversion with overflow checks, `Unmarshaler` dispatch. | | `decode.go` | Maps the parsed tree onto Go values by reflection: struct fields, maps, slices, scalar conversion with overflow checks, `Unmarshaler` dispatch. |
+41 -13
View File
@@ -1,25 +1,32 @@
# Command line # Command line
The reference below is taken from the program itself. `interpres-decode` is The reference below is taken from the program itself. `interpres-decode` is
the toml-test harness adapter, and it also validates documents. Install it the toml-test harness adapter in both directions, decoding TOML into tagged
with Go itself, no release assets involved: JSON and encoding tagged JSON back into TOML, and it also validates documents.
Install it with Go itself, no release assets involved:
```sh ```sh
go install sourcedock.dev/petrbalvin/interpres/cmd/interpres-decode@latest go install sourcedock.dev/petrbalvin/interpres/v2/cmd/interpres-decode@latest
``` ```
## Synopsis ## Synopsis
```sh ```sh
interpres-decode [flags] interpres-decode [flags]
interpres-decode -encode
interpres-decode -validate [file ...] interpres-decode -validate [file ...]
``` ```
Without `-validate` the program is the toml-test adapter: it takes no Without `-validate` or `-encode` the program is the decoding half of the
arguments, reads one TOML document from stdin, and writes the toml-test toml-test adapter: it takes no arguments, reads one TOML document from stdin,
tagged-JSON form to stdout. Build it locally with `just build`, which and writes the toml-test tagged-JSON form to stdout. Build it locally with
compiles it into `bin/interpres-decode`, or run it straight from the module `just build`, which compiles it into `bin/interpres-decode`, or run it
directory with `just run`. straight from the module directory with `just run`.
With `-encode` the direction is reversed: the program reads a tagged-JSON
description from stdin and writes the TOML document it describes to stdout,
which is the shape toml-test expects of an encoder command. It takes no
arguments either, and `-validate` and `-encode` cannot be combined.
With `-validate` the program parses each named file instead, or stdin when no With `-validate` the program parses each named file instead, or stdin when no
file is named, and prints one line per invalid document to stderr. It is file is named, and prints one line per invalid document to stderr. It is
@@ -31,15 +38,16 @@ means stdin.
| Flag | Effect | | Flag | Effect |
|---|---| |---|---|
| `-validate` | validate the documents instead of emitting tagged JSON | | `-validate` | validate the documents instead of emitting tagged JSON |
| `-encode` | read tagged JSON from stdin and write TOML instead |
| `-h` | print the usage | | `-h` | print the usage |
## Exit codes ## Exit codes
| Code | Meaning | | Code | Meaning |
|---|---| |---|---|
| `0` | adapter: the document parsed and the tagged JSON was written; validate: every document parsed | | `0` | adapter: the document parsed and the tagged JSON was written; encode: the TOML was written; validate: every document parsed |
| `1` | adapter: parse error; validate: at least one document is invalid | | `1` | adapter: parse error; validate: at least one document is invalid |
| `2` | a usage error, a read failure, or a value with no tagged representation | | `2` | a usage error, a read failure, malformed tagged JSON, or a value with no TOML representation |
## Wire format ## Wire format
@@ -66,6 +74,14 @@ wrapped in an object with a `type` and a `value`:
| local date | `date-local` | `1979-05-27` | | local date | `date-local` | `1979-05-27` |
| local time | `time-local` | `07:32:00.999999` | | local time | `time-local` | `07:32:00.999999` |
The `-encode` mode reads exactly this form back. Two properties of it are
worth knowing. A float whose value has no fraction and no exponent is written
as a bare integer string, `{"type": "float", "value": "1"}`, so there the tag
decides the type and not the literal. And the form cannot tell an array of
tables from a value array of inline tables, so the adapter writes the header
form, `[[a]]`, for an array whose every element is a JSON object; a mixed
array keeps the value form.
## Examples ## Examples
Echo a small document through the adapter: Echo a small document through the adapter:
@@ -78,8 +94,18 @@ port = 9090
' | ./bin/interpres-decode ' | ./bin/interpres-decode
``` ```
The output is the equivalent value tree as one JSON object. Validate the The output is the equivalent value tree as one JSON object. Turn a description
TOML files of another repository in CI: back into TOML with `-encode`:
```sh
echo '{"title": {"type": "string", "value": "hello"}}' | ./bin/interpres-decode -encode
```
```toml
title = "hello"
```
Validate the TOML files of another repository in CI:
```sh ```sh
interpres-decode -validate config.toml deploy/example.toml interpres-decode -validate config.toml deploy/example.toml
@@ -101,5 +127,7 @@ just toml-test
``` ```
That recipe needs the `toml-test` binary on `PATH`, installed with That recipe needs the `toml-test` binary on `PATH`, installed with
`go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0`. The full `go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0`. It runs
the suite in both directions: the decoder against the valid and invalid
corpora, and the encoder against the tagged JSON of the valid one. The full
reference for the library itself is [API.md](API.md). reference for the library itself is [API.md](API.md).
+1 -1
View File
@@ -41,7 +41,7 @@ prints the same list.
| `just run` | `go run ./cmd/interpres-decode`, reads TOML from stdin | | `just run` | `go run ./cmd/interpres-decode`, reads TOML from stdin |
| `just dev` | the same run, for iterating | | `just dev` | the same run, for iterating |
| `just example` | `go run ./examples/basic`, the usage tour | | `just example` | `go run ./examples/basic`, the usage tour |
| `just toml-test` | builds the adapter and runs the official toml-test compliance suite against it | | `just toml-test` | builds the adapter and runs the official toml-test compliance suite against it, decoder and encoder |
| `just coverage-html` | `just test`, then `go tool cover -html` into `coverage.html` | | `just coverage-html` | `just test`, then `go tool cover -html` into `coverage.html` |
| `just install` | builds, then copies the binary into `~/.local/bin` (`BINDIR` overrides) | | `just install` | builds, then copies the binary into `~/.local/bin` (`BINDIR` overrides) |
| `just uninstall` | removes the installed binary | | `just uninstall` | removes the installed binary |
+92 -1
View File
@@ -6,6 +6,7 @@ package interpres
import ( import (
"bytes" "bytes"
"context" "context"
"encoding"
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
@@ -23,6 +24,8 @@ var (
localDateType = reflect.TypeFor[LocalDate]() localDateType = reflect.TypeFor[LocalDate]()
localTimeType = reflect.TypeFor[LocalTime]() localTimeType = reflect.TypeFor[LocalTime]()
timeGoType = reflect.TypeFor[time.Time]() timeGoType = reflect.TypeFor[time.Time]()
durationType = reflect.TypeFor[time.Duration]()
textMarshalerType = reflect.TypeFor[encoding.TextMarshaler]()
) )
// encoder produces a TOML document from a Go value via a small intermediate // encoder produces a TOML document from a Go value via a small intermediate
@@ -319,6 +322,15 @@ func addField(doc *tomlDoc, name string, v reflect.Value, ctx string) error {
v = reflect.ValueOf(mv) v = reflect.ValueOf(mv)
} }
} }
// A type that renders itself as text becomes a TOML string, whether it is
// a scalar kind or a struct.
s, isText, err := textValue(v)
if err != nil {
return &EncodeError{Path: joinKey(ctx, name), Err: err}
}
if isText {
return doc.appendScalar(name, s, ctx)
}
v = followPtr(v) v = followPtr(v)
if !v.IsValid() { if !v.IsValid() {
return nil return nil
@@ -500,6 +512,20 @@ func normaliseValue(v reflect.Value) (any, error) {
if t := v.Type(); t == timeGoType || isLocalDateType(t) { if t := v.Type(); t == timeGoType || isLocalDateType(t) {
return v.Interface(), nil return v.Interface(), nil
} }
// TOML has no duration type, so a duration goes out in its canonical Go
// form, the shape it comes back in.
if v.Type() == durationType {
return time.Duration(v.Int()).String(), nil
}
// A type that renders itself as text becomes a TOML string, scalar kinds
// and structs alike.
s, isText, err := textValue(v)
if err != nil {
return nil, err
}
if isText {
return s, nil
}
switch v.Kind() { switch v.Kind() {
case reflect.String: case reflect.String:
return v.String(), nil return v.String(), nil
@@ -573,10 +599,75 @@ func isLocalDateType(t reflect.Type) bool {
return t == localDateTimeType || t == localDateType || t == localTimeType return t == localDateTimeType || t == localDateType || t == localTimeType
} }
// isDateTimeType reports whether t is one of the four TOML date-time types,
// which the encoder emits as bare atoms. Pointers are looked through. The types
// carry time.Time's text methods through an embedded field, and the atom form
// takes precedence over them.
func isDateTimeType(t reflect.Type) bool {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
return t == timeGoType || isLocalDateType(t)
}
// isTextMarshalerType reports whether t or *t implements
// encoding.TextMarshaler. An array of such values stays a value array, because
// each element's TOML form is a string.
func isTextMarshalerType(t reflect.Type) bool {
if isDateTimeType(t) {
return false
}
return t.Implements(textMarshalerType) || reflect.PointerTo(t).Implements(textMarshalerType)
}
// textValue returns the string a value renders itself as through
// encoding.TextMarshaler. The date-time types are excluded, because their
// embedded time.Time would answer with an RFC 3339 string where the TOML form
// is a bare timestamp. A nil pointer offers no text and is left to the ordinary
// nil handling, which omits the field.
func textValue(v reflect.Value) (string, bool, error) {
for v.Kind() == reflect.Interface && !v.IsNil() {
v = v.Elem()
}
if !v.IsValid() || isDateTimeType(v.Type()) {
return "", false, nil
}
if v.Kind() == reflect.Pointer && v.IsNil() {
return "", false, nil
}
m, ok := textMarshalerOf(v)
if !ok {
return "", false, nil
}
b, err := m.MarshalText()
if err != nil {
return "", true, err
}
return string(b), true, nil
}
// textMarshalerOf finds the encoding.TextMarshaler for v: on the value itself,
// or on its address, so a pointer-receiver MarshalText is found on an
// addressable struct field.
func textMarshalerOf(v reflect.Value) (encoding.TextMarshaler, bool) {
if !v.CanInterface() {
return nil, false
}
if m, ok := v.Interface().(encoding.TextMarshaler); ok {
return m, true
}
if v.CanAddr() {
if m, ok := v.Addr().Interface().(encoding.TextMarshaler); ok {
return m, true
}
}
return nil, false
}
func isTableElementType(t reflect.Type) bool { func isTableElementType(t reflect.Type) bool {
switch t.Kind() { switch t.Kind() {
case reflect.Struct: case reflect.Struct:
return !isScalarStruct(t) return !isScalarStruct(t) && !isTextMarshalerType(t)
case reflect.Map: case reflect.Map:
return t.Key().Kind() == reflect.String return t.Key().Kind() == reflect.String
} }
+200
View File
@@ -8,6 +8,7 @@ import (
"context" "context"
"errors" "errors"
"math" "math"
"net"
"reflect" "reflect"
"strings" "strings"
"testing" "testing"
@@ -1372,3 +1373,202 @@ func TestEncodeErrorHeterogeneousArrayPath(t *testing.T) {
t.Fatalf("Path = %q, want %q", ee.Path, "items[0]") t.Fatalf("Path = %q, want %q", ee.Path, "items[0]")
} }
} }
// --- encoding.TextMarshaler and time.Duration ------------------------------
// textTag is a value-receiver encoding.TextMarshaler, so the encoder finds the
// method on the value itself.
type textTag string
func (t textTag) MarshalText() ([]byte, error) { return []byte("tag:" + string(t)), nil }
// textPointer carries MarshalText on the pointer receiver only, so the encoder
// has to look at the address of an addressable field.
type textPointer struct{ V string }
func (t *textPointer) MarshalText() ([]byte, error) { return []byte(strings.ToUpper(t.V)), nil }
// textAndTOML implements both encoding interfaces; the TOML method wins.
type textAndTOML struct{}
func (textAndTOML) MarshalTOML() (any, error) { return "toml", nil }
func (textAndTOML) MarshalText() ([]byte, error) { return []byte("text"), nil }
// brokenText fails the marshal from MarshalText.
type brokenText struct{}
func (brokenText) MarshalText() ([]byte, error) { return nil, errors.New("text boom") }
// notUTF8 renders bytes that no TOML string can carry.
type notUTF8 struct{}
func (notUTF8) MarshalText() ([]byte, error) { return []byte{0xff, 0xfe}, nil }
// textTagBoth renders itself with a prefix and strips it again on decode, so
// the round trip through a TOML string is lossless.
type textTagBoth string
func (t textTagBoth) MarshalText() ([]byte, error) { return []byte("tag:" + string(t)), nil }
func (t *textTagBoth) UnmarshalText(text []byte) error {
trimmed, ok := strings.CutPrefix(string(text), "tag:")
if !ok {
return errors.New("textTagBoth: missing the tag prefix")
}
*t = textTagBoth(trimmed)
return nil
}
func TestMarshalTextValues(t *testing.T) {
// The pointer receiver is reachable only through an addressable field, so
// the whole value is marshalled through a pointer here.
type Cfg struct {
IP net.IP `toml:"ip"`
Duration time.Duration `toml:"duration"`
Tag textTag `toml:"tag"`
Pointer textPointer `toml:"pointer"`
Both textAndTOML `toml:"both"`
}
out, err := Marshal(&Cfg{
IP: net.IPv4(192, 0, 2, 1),
Duration: 90 * time.Minute,
Tag: "x",
Pointer: textPointer{V: "abc"},
})
if err != nil {
t.Fatalf("marshal: %v", err)
}
want := "ip = \"192.0.2.1\"\nduration = \"1h30m0s\"\ntag = \"tag:x\"\npointer = \"ABC\"\nboth = \"toml\"\n"
if string(out) != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
}
}
func TestMarshalTextValuesInContainers(t *testing.T) {
// Slice elements are addressable, so a pointer-receiver MarshalText is used
// there too, and an array of such values stays a value array: each element's
// TOML form is a string, so the [[header]] form cannot carry it.
type Cfg struct {
Map map[string]net.IP `toml:"map"`
Durs []time.Duration `toml:"durs"`
Ptrs []textPointer `toml:"ptrs"`
Empty []textPointer `toml:"empty"`
}
out, err := Marshal(Cfg{
Map: map[string]net.IP{"a": net.IPv4(10, 0, 0, 1)},
Durs: []time.Duration{0, 250 * time.Millisecond},
Ptrs: []textPointer{{V: "a"}, {V: "b"}},
Empty: []textPointer{},
})
if err != nil {
t.Fatalf("marshal: %v", err)
}
want := "durs = [\"0s\", \"250ms\"]\nptrs = [\"A\", \"B\"]\nempty = []\n\n[map]\na = \"10.0.0.1\"\n"
if string(out) != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
}
}
func TestMarshalTextLeavesDateTimesAlone(t *testing.T) {
// The four date-time types carry time.Time's text methods through an
// embedded field; their TOML form is a bare atom, never a quoted string.
stamp := time.Date(2026, 6, 26, 10, 0, 0, 0, time.UTC)
type Cfg struct {
Stamp time.Time `toml:"stamp"`
Ptr *time.Time `toml:"ptr"`
Day LocalDate `toml:"day"`
At LocalDateTime `toml:"at"`
Clock LocalTime `toml:"clock"`
}
out, err := Marshal(&Cfg{
Stamp: stamp,
Ptr: &stamp,
Day: LocalDate{time.Date(1979, 5, 27, 0, 0, 0, 0, time.UTC)},
At: LocalDateTime{time.Date(1979, 5, 27, 7, 32, 0, 0, time.UTC)},
Clock: LocalTime{time.Date(0, 1, 1, 7, 32, 0, 0, time.UTC)},
})
if err != nil {
t.Fatalf("marshal: %v", err)
}
want := "stamp = 2026-06-26T10:00:00Z\nptr = 2026-06-26T10:00:00Z\nday = 1979-05-27\nat = 1979-05-27T07:32:00\nclock = 07:32:00\n"
if string(out) != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
}
}
func TestMarshalTextNilPointerOmitted(t *testing.T) {
type Cfg struct {
P *textPointer `toml:"p"`
K string `toml:"k"`
}
out, err := Marshal(&Cfg{K: "x"})
if err != nil {
t.Fatalf("marshal: %v", err)
}
if want := "k = \"x\"\n"; string(out) != want {
t.Errorf("output mismatch:\ngot: %q\nwant: %q", out, want)
}
}
func TestMarshalTextErrorCarriesPath(t *testing.T) {
type Inner struct {
F brokenText `toml:"f"`
}
type Cfg struct {
Inner Inner `toml:"inner"`
}
_, err := Marshal(Cfg{})
if err == nil {
t.Fatal("expected an error from MarshalText")
}
if !strings.Contains(err.Error(), "text boom") {
t.Errorf("err = %v, want substring \"text boom\"", err)
}
ee, ok := errors.AsType[*EncodeError](err)
if !ok {
t.Fatalf("expected an *EncodeError, got %T: %v", err, err)
}
if ee.Path != "inner.f" {
t.Fatalf("Path = %q, want %q", ee.Path, "inner.f")
}
}
func TestMarshalTextRejectsInvalidUTF8(t *testing.T) {
// A TOML string holds UTF-8 only, so text that is not gets an error rather
// than replacement characters.
_, err := Marshal(struct {
V notUTF8 `toml:"v"`
}{})
if err == nil {
t.Fatal("expected an error for text that is not valid UTF-8")
}
if !strings.Contains(err.Error(), "UTF-8") {
t.Errorf("err = %v, want a UTF-8 message", err)
}
}
func TestMarshalTextValuesRoundTrip(t *testing.T) {
type Cfg struct {
Duration time.Duration `toml:"duration"`
IP net.IP `toml:"ip"`
Tag textTagBoth `toml:"tag"`
}
in := Cfg{Duration: 90 * time.Minute, IP: net.IPv4(198, 51, 100, 7), Tag: "y"}
out, err := Marshal(&in)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var back Cfg
if err := Unmarshal(out, &back); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if back.Duration != in.Duration {
t.Errorf("Duration = %v, want %v", back.Duration, in.Duration)
}
if !back.IP.Equal(in.IP) {
t.Errorf("IP = %v, want %v", back.IP, in.IP)
}
if back.Tag != in.Tag {
t.Errorf("Tag = %q, want %q", back.Tag, in.Tag)
}
}
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"os" "os"
"time" "time"
"sourcedock.dev/petrbalvin/interpres" "sourcedock.dev/petrbalvin/interpres/v2"
) )
// document is a small but realistic configuration: it has scalars, a // document is a small but realistic configuration: it has scalars, a
+1 -1
View File
@@ -1,3 +1,3 @@
module sourcedock.dev/petrbalvin/interpres module sourcedock.dev/petrbalvin/interpres/v2
go 1.27.1 go 1.27.1
+21 -6
View File
@@ -123,6 +123,11 @@ func ParseContext(ctx context.Context, data []byte) (map[string]any, error) {
// case-insensitive match on the field name when no tag is present. A tag of // case-insensitive match on the field name when no tag is present. A tag of
// "-" skips the field. // "-" skips the field.
// //
// A destination implementing Unmarshaler receives the parsed value as it is,
// a TOML string fills a destination implementing encoding.TextUnmarshaler, and
// a time.Duration destination takes a duration literal such as `1h30m` or a
// bare integer as its nanosecond count.
//
// Unmarshal is equivalent to UnmarshalContext with context.Background. // Unmarshal is equivalent to UnmarshalContext with context.Background.
func Unmarshal(data []byte, v any) error { func Unmarshal(data []byte, v any) error {
return UnmarshalContext(context.Background(), data, v) return UnmarshalContext(context.Background(), data, v)
@@ -177,6 +182,10 @@ func (d *Decoder) DecodeContext(ctx context.Context, data []byte, v any) error {
// then encodes as if the returned value had been passed in its place, which // then encodes as if the returned value had been passed in its place, which
// is useful for emitting a Go type as a different TOML shape (for example, a // is useful for emitting a Go type as a different TOML shape (for example, a
// struct as an inline table or a primitive alias as a richer value). // struct as an inline table or a primitive alias as a richer value).
//
// MarshalTOML wins over encoding.TextMarshaler when a type implements both.
// A type that implements only encoding.TextMarshaler is encoded as a TOML
// string holding its text, and needs no method here.
type Marshaler interface { type Marshaler interface {
MarshalTOML() (any, error) MarshalTOML() (any, error)
} }
@@ -194,12 +203,15 @@ type Marshaler interface {
// UnmarshalTOML is invoked from (*Decoder).Decode / Unmarshal when the // UnmarshalTOML is invoked from (*Decoder).Decode / Unmarshal when the
// destination type implements the interface. The decoder does not need to // destination type implements the interface. The decoder does not need to
// consult the concrete return value; whatever the receiver stores is kept. // consult the concrete return value; whatever the receiver stores is kept.
//
// UnmarshalTOML wins over encoding.TextUnmarshaler when a type implements
// both. A type that implements only encoding.TextUnmarshaler is filled from a
// TOML string holding its text, and needs no method here.
type Unmarshaler interface { type Unmarshaler interface {
UnmarshalTOML(data any) error UnmarshalTOML(data any) error
} }
// Marshal returns the TOML encoding of v. The output stays within TOML 1.0, // Marshal returns the TOML encoding of v. The output is valid TOML 1.1.
// so it is valid under both TOML 1.0 and 1.1.
// //
// Marshal traverses v using reflection and applies the following rules: // Marshal traverses v using reflection and applies the following rules:
// //
@@ -223,6 +235,9 @@ type Unmarshaler interface {
// variants). // variants).
// - Values implementing Marshaler are encoded by calling MarshalTOML and // - Values implementing Marshaler are encoded by calling MarshalTOML and
// using its result. // using its result.
// - Values implementing encoding.TextMarshaler, and not one of the
// date-time types, encode as a TOML string holding the text the method
// returns. time.Duration is written in its canonical Go form, `1h30m0s`.
// - nil pointer fields are omitted. // - nil pointer fields are omitted.
// //
// Marshal cannot encode cyclic data structures; passing one will loop until // Marshal cannot encode cyclic data structures; passing one will loop until
@@ -246,14 +261,14 @@ func MarshalContext(ctx context.Context, v any) ([]byte, error) {
// An Encoder encodes Go values into TOML. // An Encoder encodes Go values into TOML.
// //
// All options default to behaviour that preserves byte-for-byte compatibility // All options default to the behaviour earlier releases used, and the defaults
// with previous releases and passes the toml-test compliance suite: // pass the toml-test compliance suite in both directions:
// //
// GroupByKind: true (scalars first, then tables, then arrays of tables) // GroupByKind: true (scalars first, then tables, then arrays of tables)
// OmitEmptyArrays: false (a nil/empty []string slice emits [] as a value; // OmitEmptyArrays: false (a nil/empty []string slice emits [] as a value;
// a nil/empty []Item struct slice is still skipped) // a nil/empty []Item struct slice is still skipped)
// LiteralMultilineAt: 0 (always emit basic multi-line strings with // LiteralMultilineAt: 0 (always emit the escaped basic form, never a
// escape sequences, never literal ones) // literal one)
// //
// Use the chainable option methods to opt out. The option state is private; // Use the chainable option methods to opt out. The option state is private;
// callers that need the underlying knobs reach for the methods rather than // callers that need the underlying knobs reach for the methods rather than
+2 -2
View File
@@ -92,9 +92,9 @@ run:
dev: dev:
go run -buildvcs=true {{package}} go run -buildvcs=true {{package}}
# Runs the official toml-test compliance suite against the built adapter; toml-test must be on PATH (go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0); not standard because no canonical recipe covers a domain compliance suite. # Runs the official toml-test compliance suite in both directions, decoder and encoder, against the built adapter; toml-test must be on PATH (go install github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.2.0); not standard because no canonical recipe covers a domain compliance suite.
toml-test: build toml-test: build
toml-test test -decoder=bin/interpres-decode -toml=1.1 toml-test test -decoder=bin/interpres-decode -encoder='bin/interpres-decode -encode' -toml=1.1
# Coverage report as an HTML map from the gate's profile; not standard because the gate needs only the numeric floor, and a browser artefact is exploration, not a gate. # Coverage report as an HTML map from the gate's profile; not standard because the gate needs only the numeric floor, and a browser artefact is exploration, not a gate.
coverage-html: test coverage-html: test