240 lines
12 KiB
Markdown
240 lines
12 KiB
Markdown
# Architecture
|
|
|
|
**nuntius** is a small HTTP service that accepts JSON form submissions, validates them, sends them via SMTP, and (for `newsletter` forms) appends the email to a JSONL log on disk. This document explains the layers, the request flow, and the design decisions that keep the binary ~6 MB and the dependency graph empty.
|
|
|
|
## High-Level Overview
|
|
|
|
```mermaid
|
|
graph TD
|
|
Browser[Browser / Frontend]:::accent6
|
|
Nginx[nginx reverse proxy]:::accent6
|
|
Server[cmd/server/main.go]:::accent0
|
|
Config[internal/config/]:::accent7
|
|
Handler[internal/handler/]:::accent0
|
|
RateLimit[rateLimiter per form]:::accent7
|
|
CORS[CORS allowlist per form]:::accent7
|
|
Honeypot[honeypot_field per form]:::accent7
|
|
Validate[pkg/contactform/Validate]:::accent3
|
|
Sender[internal/email/FormSender]:::accent1
|
|
Storage[internal/storage/NewsletterStore]:::accent1
|
|
SMTP[(SMTP server)]:::accent2
|
|
JSONL[(data_dir/newsletter-*.jsonl)]:::accent2
|
|
|
|
Browser -->|POST + Origin| Nginx
|
|
Nginx -->|X-Forwarded-For| Server
|
|
Server --> Config
|
|
Server --> Handler
|
|
Handler --> RateLimit
|
|
Handler --> CORS
|
|
Handler --> Honeypot
|
|
Handler --> Validate
|
|
Handler --> Sender
|
|
Handler --> Storage
|
|
Sender -->|SMTP PLAIN over net/smtp| SMTP
|
|
Storage -->|append JSONL| JSONL
|
|
Handler -->|200 / 4xx / 5xx| Nginx
|
|
Nginx --> Browser
|
|
|
|
classDef accent0 fill:#3B82F6,stroke:#2563EB,color:#fff
|
|
classDef accent1 fill:#22C55E,stroke:#16A34A,color:#fff
|
|
classDef accent2 fill:#F59E0B,stroke:#D97706,color:#000
|
|
classDef accent3 fill:#A855F7,stroke:#7C3AED,color:#fff
|
|
classDef accent6 fill:#6366F1,stroke:#4F46E5,color:#fff
|
|
classDef accent7 fill:#64748B,stroke:#475569,color:#fff
|
|
```
|
|
|
|
## Layers
|
|
|
|
### 1. Entry point — `cmd/server/main.go`
|
|
|
|
- Wires up a JSON `slog` handler on stdout.
|
|
- Loads the config from `config.ConfigPath()` (default `/etc/nuntius/config.toml`).
|
|
- Builds a `*handler.ContactHandler` via `handler.New(cfg)`.
|
|
- Registers one `POST` + one `OPTIONS` handler per form on a fresh `http.ServeMux`, plus `GET /health`.
|
|
- Wraps the mux in a request-logging middleware that emits a single `slog.Info` per request with method, path, status, duration, and client IP.
|
|
- Starts an `http.Server` with explicit read / write / idle timeouts.
|
|
- Installs a `signal.NotifyContext` listener for SIGINT / SIGTERM and shuts the server down gracefully within 15 s.
|
|
|
|
The main package owns **no business logic** — it only orchestrates the four internal packages.
|
|
|
|
### 2. Configuration — `internal/config/`
|
|
|
|
The config package is the gatekeeper for everything that varies per deployment.
|
|
|
|
| Concern | Where |
|
|
|---|---|
|
|
| File path | `config.ConfigPath()` returns `/etc/nuntius/config.toml` |
|
|
| Default file | `defaultConfig` is a `[]byte` constant embedded in the source; written on first start |
|
|
| Env-var expansion | `os.Expand(string(raw), os.Getenv)` runs **before** TOML is parsed by `interpres` |
|
|
| Strict decoding | `interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos |
|
|
| Schema validation | `(*Config).validate()` enforces required fields, unique paths, valid form types, sensible defaults |
|
|
| Defaults | `port = 8080`, `data_dir = "./data"`, `rate_limit_per_hour = 10`, `honeypot_field = "website"`, `from = smtp.user` |
|
|
|
|
The `Form` type mirrors the TOML shape one-to-one. `ValidFormTypes` is the authoritative allow-list — adding a form type means editing one map in this package.
|
|
|
|
### 3. HTTP handler — `internal/handler/`
|
|
|
|
`ContactHandler` is a single struct with four maps keyed by URL path:
|
|
|
|
```go
|
|
type ContactHandler struct {
|
|
forms map[string]*config.Form
|
|
senders map[string]*email.FormSender
|
|
rateLimits map[string]*rateLimiter
|
|
stores map[string]*storage.NewsletterStore
|
|
}
|
|
```
|
|
|
|
`New(cfg)` builds all four maps in one pass. `Register(mux)` then mounts `POST <path>` and `OPTIONS <path>` per form, plus `GET /health`.
|
|
|
|
The per-form handler closure (`makeHandler`) runs this pipeline in order:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant Handler as ContactHandler
|
|
participant CORS
|
|
participant Limiter as rateLimiter
|
|
participant Body
|
|
participant Honey as Honeypot
|
|
participant Validate as contactform.Validate
|
|
participant Sender as email.FormSender
|
|
participant Store as storage.NewsletterStore
|
|
|
|
Client->>Handler: POST /api/nuntius/contact
|
|
alt preflight
|
|
Handler->>CORS: formAllowed(origin)?
|
|
CORS-->>Handler: yes / no
|
|
Handler-->>Client: 204 No Content (or 403)
|
|
else actual
|
|
Handler->>CORS: formAllowed(origin)?
|
|
CORS-->>Handler: yes / no
|
|
alt origin present and not allowed
|
|
Handler-->>Client: 403 origin_not_allowed
|
|
end
|
|
Handler->>Limiter: allow(ip)
|
|
alt over limit
|
|
Handler-->>Client: 429 rate_limited
|
|
end
|
|
Handler->>Body: json.Decode
|
|
alt invalid JSON
|
|
Handler-->>Client: 400 invalid_json
|
|
end
|
|
Handler->>Honey: honeypot field non-empty?
|
|
alt bot
|
|
Handler-->>Client: 200 ok (silent)
|
|
end
|
|
Handler->>Validate: contactform.Validate(req, form.Type)
|
|
alt errors
|
|
Handler-->>Client: 400 validation + details
|
|
end
|
|
Handler->>Sender: Send(req)
|
|
alt SMTP error
|
|
Handler-->>Client: 500 send_failed
|
|
end
|
|
opt form.Type == "newsletter"
|
|
Handler->>Store: Append(Subscriber{...})
|
|
alt write error
|
|
Handler-->>Client: 500 storage_failed
|
|
end
|
|
end
|
|
Handler-->>Client: 200 ok
|
|
end
|
|
```
|
|
|
|
Notable behaviors:
|
|
|
|
- **CORS is path-aware** — Each form has its own `allowed_origins` list; preflights and actual requests are checked against the form being hit, not the host.
|
|
- **Client IP honours reverse proxies** — `clientIP` reads `X-Forwarded-For` (first hop), then `X-Real-IP`, then falls back to `RemoteAddr`. Strip the port by hand because the stdlib does not give us a `netip.AddrPort` for the remote.
|
|
- **Rate limiting is in-process** — `rateLimiter` is a per-IP token bucket. The bucket refills at `perHour/3600` tokens per second. The map is guarded by a single `sync.Mutex`; in practice the lock is held for microseconds and the access pattern is dominated by `Allow` lookups, not insert / delete.
|
|
- **Honeypot is a positive control** — A bot that fills the invisible field gets a 200 with no email sent and no log line. Humans are never blocked.
|
|
- **Validation is the same function the public library uses** — `contactform.Validate` is the only validator in the codebase; the handler just calls it with `form.Type` and forwards the returned `[]FieldError` to the client.
|
|
|
|
### 4. Email — `internal/email/`
|
|
|
|
Each form gets a `FormSender` constructed once in `handler.New`. `Send(req)` is the only public method:
|
|
|
|
1. `smtp.PlainAuth("", user, password, host)` — the empty identity means "use the supplied user as-is", which is what every modern SMTP provider expects.
|
|
2. `compose(from, to, req, formName, formType)` builds a `multipart/alternative` message (text/plain + text/html).
|
|
3. `smtp.SendMail(addr, auth, from, []string{to}, msg)`.
|
|
|
|
`compose` picks one of four HTML templates (`contact`, `feedback`, `newsletter`, `generic`) and four matching plain-text subjects. The subject for `contact` includes the `[<service>]` tag if `req.Service` is set, so inbox filters can route by service interest.
|
|
|
|
Credentials are never logged — neither the password nor the full `From` / `To` headers appear in any `slog` line. The handler logs `form`, `path`, `service` (if any), and `ip` only.
|
|
|
|
### 5. Storage — `internal/storage/`
|
|
|
|
`NewsletterStore` is a thin wrapper over an append-only JSONL file:
|
|
|
|
| Method | Semantics |
|
|
|---|---|
|
|
| `Append(Subscriber)` | Creates the parent dir if missing, opens the file with `O_APPEND \| O_CREATE \| O_WRONLY`, writes one JSON line + `\n`, closes. Sets `CreatedAt = time.Now().UTC()` if zero. |
|
|
| `Count()` | Streams the file with a `bufio.Scanner` (1 MB max line), increments on each line that unmarshals into a non-empty `Subscriber`. Malformed lines are skipped. |
|
|
| `List()` | Same scan, returns all valid subscribers in insertion order. Reads the entire log into memory — do not call on multi-million-row files. |
|
|
| `Path()` | Returns the configured file path. |
|
|
|
|
A single `sync.Mutex` guards all three methods. The store is safe for concurrent use from the handler goroutines.
|
|
|
|
There is **no rotation**. Add a `logrotate` unit if the file grows beyond what you want to `wc -l`.
|
|
|
|
## Request Flow (single form)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Browser
|
|
participant Nginx
|
|
participant Nuntius as nuntius
|
|
participant SMTP
|
|
participant Disk
|
|
|
|
Browser->>Nginx: POST /api/nuntius/contact
|
|
Nginx->>Nuntius: forward (X-Forwarded-For, X-Real-IP)
|
|
Nuntius->>Nuntius: CORS check
|
|
Nuntius->>Nuntius: rate-limit per IP
|
|
Nuntius->>Nuntius: parse JSON
|
|
Nuntius->>Nuntius: honeypot check
|
|
Nuntius->>Nuntius: contactform.Validate
|
|
Nuntius->>SMTP: SMTP PLAIN auth + multipart message
|
|
SMTP-->>Nuntius: 250 OK
|
|
opt newsletter form
|
|
Nuntius->>Disk: append one JSONL line
|
|
end
|
|
Nuntius-->>Nginx: 200 {"ok": true}
|
|
Nginx-->>Browser: 200 {"ok": true}
|
|
```
|
|
|
|
## Config Merging
|
|
|
|
There is no merging in nuntius — the TOML file is the only source of runtime configuration. The file itself goes through three stages at startup:
|
|
|
|
1. **Default generation** — If the file does not exist, `os.WriteFile(path, defaultConfig, 0644)` writes the embedded three-form template.
|
|
2. **Env-var expansion** — `os.Expand` substitutes `${VAR}` and `$VAR` references **in the raw text** before parsing. The parsed struct never holds a literal like `"${NUNTIUS_SMTP_PASSWORD}"` — it holds the expanded value (or an empty string if the env var was unset).
|
|
3. **Strict parsing + validation** — `interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos; `validate()` enforces required fields, unique paths, valid form types, and applies defaults.
|
|
|
|
The order matters: expansion before parsing means env-var references work even for values that are not strings (e.g. an integer with a `${PORT}` reference would fail, but you can still reference strings).
|
|
|
|
## Error Handling
|
|
|
|
Errors are returned as a typed string in the JSON body, with the HTTP status code doing the heavy lifting:
|
|
|
|
| Status | Body shape | When |
|
|
|---|---|---|
|
|
| `200` | `{"ok": true}` | Submission accepted; mail sent (and subscriber persisted for `newsletter`) |
|
|
| `200` | `{"ok": true}` | Honeypot triggered (silent accept, no mail) |
|
|
| `400` | `{"error": "invalid_json", "message": "..."}` | Body is not valid JSON |
|
|
| `400` | `{"error": "validation", "details": [...]}` | One or more fields are invalid |
|
|
| `403` | `{"error": "origin_not_allowed"}` | `Origin` header is not in the form's `allowed_origins` |
|
|
| `429` | `{"error": "rate_limited", "message": "..."}` | Token bucket empty for this IP / form |
|
|
| `500` | `{"error": "send_failed", "message": "..."}` | SMTP round-trip failed |
|
|
| `500` | `{"error": "storage_failed", "message": "..."}` | `newsletter` only — could not write the JSONL line |
|
|
|
|
The `message` field is intentionally generic; field-level details for validation errors live in `details[]`. None of the error bodies leak SMTP credentials, file paths, or stack traces.
|
|
|
|
## Why These Choices
|
|
|
|
- **stdlib only** — `net/smtp`, `net/http`, `net/mail`, `log/slog`, `bufio`, `os/signal`, `sync` plus the first-party `interpres` TOML library are all that is needed. The build is reproducible, the binary is small, and the external supply chain is empty.
|
|
- **No global state outside the binary** — `ContactHandler` owns the maps; `slog` is the only package-level default. Everything else is constructed explicitly in `New(cfg)` so the handler is easy to instantiate from tests.
|
|
- **Append-only JSONL** — Cheaper to implement than SQLite, easier to inspect than a custom binary format, easy to back up, and resilient to partial writes.
|
|
- **Per-form isolation** — One form's SMTP credentials never travel through another form's code path. A misconfigured `contact` form cannot leak credentials from a `newsletter` form.
|
|
- **Strict config parsing** — Catching `forms[0].smtp.port = "587"` (string) or `forms[0].type = "contatc"` (typo) at startup is worth the small extra error-path code.
|