Files

258 lines
13 KiB
Markdown
Raw Permalink Normal View History

# Architecture
**nuntius** is a small HTTP service that accepts JSON form submissions, validates them, sends them via SMTP, and (for `newsletter` forms) appends the email to a JSONL log on disk. This document explains the layers, the request flow, and the design decisions that keep the binary ~6 MB and the dependency graph empty.
## High-Level Overview
```mermaid
graph TD
Browser[Browser / Frontend]:::accent6
Caddy[Caddy reverse proxy]:::accent6
Server[cmd/server/main.go]:::accent0
Config[internal/config/]:::accent7
Handler[internal/handler/]:::accent0
RateLimit[rateLimiter per form]:::accent7
CORS[CORS allowlist per form]:::accent7
Honeypot[honeypot_field per form]:::accent7
Validate[pkg/contactform/Validate]:::accent3
Sender[internal/email/FormSender]:::accent1
Storage[internal/storage/NewsletterStore]:::accent1
SMTP[(SMTP server)]:::accent2
JSONL[(data_dir/newsletter-*.jsonl)]:::accent2
Browser -->|POST + Origin| Caddy
Caddy -->|X-Forwarded-For| Server
Server --> Config
Server --> Handler
Handler --> RateLimit
Handler --> CORS
Handler --> Honeypot
Handler --> Validate
Handler --> Sender
Handler --> Storage
Sender -->|SMTP PLAIN over net/smtp| SMTP
Storage -->|append JSONL| JSONL
Handler -->|200 / 4xx / 5xx| Caddy
Caddy --> Browser
classDef accent0 fill:#3B82F6,stroke:#2563EB,color:#fff
classDef accent1 fill:#22C55E,stroke:#16A34A,color:#fff
classDef accent2 fill:#F59E0B,stroke:#D97706,color:#000
classDef accent3 fill:#A855F7,stroke:#7C3AED,color:#fff
classDef accent6 fill:#6366F1,stroke:#4F46E5,color:#fff
classDef accent7 fill:#64748B,stroke:#475569,color:#fff
```
## Layers
### 1. Entry point — `cmd/server/main.go`
- Wires up a JSON `slog` handler on stdout.
- Loads the config from `config.ConfigPath()` (default `/etc/nuntius/config.toml`).
- Builds a `*handler.ContactHandler` via `handler.New(cfg)`.
- Registers one `POST` + one `OPTIONS` handler per form on a fresh `http.ServeMux`, plus `GET /health`.
- Wraps the mux in a request-logging middleware that emits a single `slog.Info` per request with method, path, status, duration, and client IP.
- Starts an `http.Server` with explicit read / write / idle timeouts.
- Installs a `signal.NotifyContext` listener for SIGINT / SIGTERM and shuts the server down gracefully within 15 s.
The main package owns **no business logic** — it only orchestrates the four internal packages.
### 2. Configuration — `internal/config/`
The config package is the gatekeeper for everything that varies per deployment.
| Concern | Where |
|---|---|
| File path | `config.ConfigPath()` returns `/etc/nuntius/config.toml` |
| Default file | `defaultConfig` is a `[]byte` constant embedded in the source; written on first start |
| Env-var expansion | `os.Expand(string(raw), os.Getenv)` runs **before** TOML is parsed by `interpres` |
| Strict decoding | `interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos |
| Schema validation | `(*Config).validate()` enforces required fields, unique paths, valid form types, sensible defaults |
| Defaults | `port = 8080`, `data_dir = "./data"`, `rate_limit_per_hour = 10`, `honeypot_field = "website"`, `from = smtp.user` |
The `Form` type mirrors the TOML shape one-to-one. `ValidFormTypes` is the authoritative allow-list — adding a form type means editing one map in this package.
### 3. HTTP handler — `internal/handler/`
`ContactHandler` stores its dependencies behind small interfaces so the handler pipeline can be tested without a real SMTP server or filesystem:
```go
type formSender interface {
Send(req contactform.Request) error
}
type subscriberStorer interface {
Append(sub storage.Subscriber) error
}
```
`email.FormSender` and `storage.NewsletterStore` satisfy these interfaces in production. Tests use mock implementations (`mockSender`, `mockStore`) that record calls and simulate errors.
`ContactHandler` is a single struct with four maps keyed by URL path:
```go
type ContactHandler struct {
forms map[string]*config.Form
senders map[string]formSender
rateLimits map[string]*rateLimiter
stores map[string]subscriberStorer
}
```
`New(cfg)` builds all four maps in one pass. `Register(mux)` then mounts `POST <path>` and `OPTIONS <path>` per form, plus `GET /health`.
The per-form handler closure (`makeHandler`) runs this pipeline in order:
```mermaid
sequenceDiagram
participant Client
participant Handler as ContactHandler
participant CORS
participant Limiter as rateLimiter
participant Body
participant Honey as Honeypot
participant Validate as contactform.Validate
participant Sender as email.FormSender
participant Store as storage.NewsletterStore
Client->>Handler: POST /api/nuntius/contact
alt preflight
Handler->>CORS: formAllowed(origin)?
CORS-->>Handler: yes / no
Handler-->>Client: 204 No Content (or 403)
else actual
Handler->>CORS: formAllowed(origin)?
CORS-->>Handler: yes / no
alt origin present and not allowed
Handler-->>Client: 403 origin_not_allowed
end
Handler->>Limiter: allow(ip)
alt over limit
Handler-->>Client: 429 rate_limited
end
Handler->>Body: json.Decode
alt invalid JSON
Handler-->>Client: 400 invalid_json
end
Handler->>Honey: honeypot field non-empty?
alt bot
Handler-->>Client: 200 ok (silent)
end
Handler->>Validate: contactform.Validate(req, form.Type)
alt errors
Handler-->>Client: 400 validation + details
end
Handler->>Sender: Send(req)
alt SMTP error
Handler-->>Client: 500 send_failed
end
opt form.Type == "newsletter"
Handler->>Store: Append(Subscriber{...})
alt write error
Handler-->>Client: 500 storage_failed
end
end
Handler-->>Client: 200 ok
end
```
### Rate limiter cleanup
The per-form `rateLimiter` type starts a background goroutine (`startCleanup`) that ticks once per hour and deletes IP entries older than twice the refill window. This prevents unbounded memory growth from one-off IPs. The cleanup goroutine is covered by `TestRateLimiterCleanup` in `contact_test.go`.
Notable behaviors:
- **CORS is path-aware** — Each form has its own `allowed_origins` list; preflights and actual requests are checked against the form being hit, not the host.
- **Client IP honours reverse proxies** — `clientIP` reads `X-Forwarded-For` (first hop), then `X-Real-IP`, then falls back to `RemoteAddr`. Strip the port by hand because the stdlib does not give us a `netip.AddrPort` for the remote.
- **Rate limiting is in-process** — `rateLimiter` is a per-IP token bucket. The bucket refills at `perHour/3600` tokens per second. The map is guarded by a single `sync.Mutex`; in practice the lock is held for microseconds and the access pattern is dominated by `Allow` lookups, not insert / delete.
- **Honeypot is a positive control** — A bot that fills the invisible field gets a 200 with no email sent and no log line. Humans are never blocked.
- **Validation is the same function the public library uses** — `contactform.Validate` is the only validator in the codebase; the handler just calls it with `form.Type` and forwards the returned `[]FieldError` to the client.
### 4. Email — `internal/email/`
Each form gets a `FormSender` constructed once in `handler.New`. `Send(req)` is the only public method:
1. `smtp.PlainAuth("", user, password, host)` — the empty identity means "use the supplied user as-is", which is what every modern SMTP provider expects.
2. `compose(from, to, req, formName, formType)` builds a `multipart/alternative` message (text/plain + text/html).
3. `smtp.SendMail(addr, auth, from, []string{to}, msg)`.
`compose` picks one of four HTML templates (`contact`, `feedback`, `newsletter`, `generic`) and four matching plain-text subjects. The subject for `contact` includes the `[<service>]` tag if `req.Service` is set, so inbox filters can route by service interest.
Credentials are never logged — neither the password nor the full `From` / `To` headers appear in any `slog` line. The handler logs `form`, `path`, `service` (if any), and `ip` only.
### 5. Storage — `internal/storage/`
`NewsletterStore` is a thin wrapper over an append-only JSONL file:
| Method | Semantics |
|---|---|
| `Append(Subscriber)` | Creates the parent dir if missing, opens the file with `O_APPEND \| O_CREATE \| O_WRONLY`, writes one JSON line + `\n`, closes. Sets `CreatedAt = time.Now().UTC()` if zero. |
| `Count()` | Streams the file with a `bufio.Scanner` (1 MB max line), increments on each line that unmarshals into a non-empty `Subscriber`. Malformed lines are skipped. |
| `List()` | Same scan, returns all valid subscribers in insertion order. Reads the entire log into memory — do not call on multi-million-row files. |
| `Path()` | Returns the configured file path. |
A single `sync.Mutex` guards all three methods. The store is safe for concurrent use from the handler goroutines.
There is **no rotation**. Add a `logrotate` unit if the file grows beyond what you want to `wc -l`.
## Request Flow (single form)
```mermaid
sequenceDiagram
participant Browser
participant Caddy
participant Nuntius as nuntius
participant SMTP
participant Disk
Browser->>Caddy: POST /api/nuntius/contact
Caddy->>Nuntius: forward (X-Forwarded-For, X-Real-IP)
Nuntius->>Nuntius: CORS check
Nuntius->>Nuntius: rate-limit per IP
Nuntius->>Nuntius: parse JSON
Nuntius->>Nuntius: honeypot check
Nuntius->>Nuntius: contactform.Validate
Nuntius->>SMTP: SMTP PLAIN auth + multipart message
SMTP-->>Nuntius: 250 OK
opt newsletter form
Nuntius->>Disk: append one JSONL line
end
Nuntius-->>Caddy: 200 {"ok": true}
Caddy-->>Browser: 200 {"ok": true}
```
## Config Merging
There is no merging in nuntius — the TOML file is the only source of runtime configuration. The file itself goes through three stages at startup:
1. **Default generation** — If the file does not exist, `os.WriteFile(path, defaultConfig, 0644)` writes the embedded three-form template.
2. **Env-var expansion**`os.Expand` substitutes `${VAR}` and `$VAR` references **in the raw text** before parsing. The parsed struct never holds a literal like `"${NUNTIUS_SMTP_PASSWORD}"` — it holds the expanded value (or an empty string if the env var was unset).
3. **Strict parsing + validation**`interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos; `validate()` enforces required fields, unique paths, valid form types, and applies defaults.
The order matters: expansion before parsing means env-var references work even for values that are not strings (e.g. an integer with a `${PORT}` reference would fail, but you can still reference strings).
## Error Handling
Errors are returned as a typed string in the JSON body, with the HTTP status code doing the heavy lifting:
| Status | Body shape | When |
|---|---|---|
| `200` | `{"ok": true}` | Submission accepted; mail sent (and subscriber persisted for `newsletter`) |
| `200` | `{"ok": true}` | Honeypot triggered (silent accept, no mail) |
| `400` | `{"error": "invalid_json", "message": "..."}` | Body is not valid JSON |
| `400` | `{"error": "validation", "details": [...]}` | One or more fields are invalid |
| `403` | `{"error": "origin_not_allowed"}` | `Origin` header is not in the form's `allowed_origins` |
| `429` | `{"error": "rate_limited", "message": "..."}` | Token bucket empty for this IP / form |
| `500` | `{"error": "send_failed", "message": "..."}` | SMTP round-trip failed |
| `500` | `{"error": "storage_failed", "message": "..."}` | `newsletter` only — could not write the JSONL line |
The `message` field is intentionally generic; field-level details for validation errors live in `details[]`. None of the error bodies leak SMTP credentials, file paths, or stack traces.
## Why These Choices
- **stdlib only** — `net/smtp`, `net/http`, `net/mail`, `log/slog`, `bufio`, `os/signal`, `sync` plus the first-party `interpres` TOML library are all that is needed. The build is reproducible, the binary is small, and the external supply chain is empty.
- **No global state outside the binary** — `ContactHandler` owns the maps; `slog` is the only package-level default. Everything else is constructed explicitly in `New(cfg)` so the handler is easy to instantiate from tests.
- **Append-only JSONL** — Cheaper to implement than SQLite, easier to inspect than a custom binary format, easy to back up, and resilient to partial writes.
- **Per-form isolation** — One form's SMTP credentials never travel through another form's code path. A misconfigured `contact` form cannot leak credentials from a `newsletter` form.
- **Strict config parsing** — Catching `forms[0].smtp.port = "587"` (string) or `forms[0].type = "contatc"` (typo) at startup is worth the small extra error-path code.