feat: initialize nuntius project with full server implementation
This commit is contained in:
@@ -0,0 +1,239 @@
|
||||
# Architecture
|
||||
|
||||
**nuntius** is a small HTTP service that accepts JSON form submissions, validates them, sends them via SMTP, and (for `newsletter` forms) appends the email to a JSONL log on disk. This document explains the layers, the request flow, and the design decisions that keep the binary ~6 MB and the dependency graph empty.
|
||||
|
||||
## High-Level Overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Browser[Browser / Frontend]:::accent6
|
||||
Nginx[nginx reverse proxy]:::accent6
|
||||
Server[cmd/server/main.go]:::accent0
|
||||
Config[internal/config/]:::accent7
|
||||
Handler[internal/handler/]:::accent0
|
||||
RateLimit[rateLimiter per form]:::accent7
|
||||
CORS[CORS allowlist per form]:::accent7
|
||||
Honeypot[honeypot_field per form]:::accent7
|
||||
Validate[pkg/contactform/Validate]:::accent3
|
||||
Sender[internal/email/FormSender]:::accent1
|
||||
Storage[internal/storage/NewsletterStore]:::accent1
|
||||
SMTP[(SMTP server)]:::accent2
|
||||
JSONL[(data_dir/newsletter-*.jsonl)]:::accent2
|
||||
|
||||
Browser -->|POST + Origin| Nginx
|
||||
Nginx -->|X-Forwarded-For| Server
|
||||
Server --> Config
|
||||
Server --> Handler
|
||||
Handler --> RateLimit
|
||||
Handler --> CORS
|
||||
Handler --> Honeypot
|
||||
Handler --> Validate
|
||||
Handler --> Sender
|
||||
Handler --> Storage
|
||||
Sender -->|SMTP PLAIN over net/smtp| SMTP
|
||||
Storage -->|append JSONL| JSONL
|
||||
Handler -->|200 / 4xx / 5xx| Nginx
|
||||
Nginx --> Browser
|
||||
|
||||
classDef accent0 fill:#3B82F6,stroke:#2563EB,color:#fff
|
||||
classDef accent1 fill:#22C55E,stroke:#16A34A,color:#fff
|
||||
classDef accent2 fill:#F59E0B,stroke:#D97706,color:#000
|
||||
classDef accent3 fill:#A855F7,stroke:#7C3AED,color:#fff
|
||||
classDef accent6 fill:#6366F1,stroke:#4F46E5,color:#fff
|
||||
classDef accent7 fill:#64748B,stroke:#475569,color:#fff
|
||||
```
|
||||
|
||||
## Layers
|
||||
|
||||
### 1. Entry point — `cmd/server/main.go`
|
||||
|
||||
- Wires up a JSON `slog` handler on stdout.
|
||||
- Loads the config from `config.ConfigPath()` (default `/etc/nuntius/config.toml`).
|
||||
- Builds a `*handler.ContactHandler` via `handler.New(cfg)`.
|
||||
- Registers one `POST` + one `OPTIONS` handler per form on a fresh `http.ServeMux`, plus `GET /health`.
|
||||
- Wraps the mux in a request-logging middleware that emits a single `slog.Info` per request with method, path, status, duration, and client IP.
|
||||
- Starts an `http.Server` with explicit read / write / idle timeouts.
|
||||
- Installs a `signal.NotifyContext` listener for SIGINT / SIGTERM and shuts the server down gracefully within 15 s.
|
||||
|
||||
The main package owns **no business logic** — it only orchestrates the four internal packages.
|
||||
|
||||
### 2. Configuration — `internal/config/`
|
||||
|
||||
The config package is the gatekeeper for everything that varies per deployment.
|
||||
|
||||
| Concern | Where |
|
||||
|---|---|
|
||||
| File path | `config.ConfigPath()` returns `/etc/nuntius/config.toml` |
|
||||
| Default file | `defaultConfig` is a `[]byte` constant embedded in the source; written on first start |
|
||||
| Env-var expansion | `os.Expand(string(raw), os.Getenv)` runs **before** TOML is parsed by `interpres` |
|
||||
| Strict decoding | `interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos |
|
||||
| Schema validation | `(*Config).validate()` enforces required fields, unique paths, valid form types, sensible defaults |
|
||||
| Defaults | `port = 8080`, `data_dir = "./data"`, `rate_limit_per_hour = 10`, `honeypot_field = "website"`, `from = smtp.user` |
|
||||
|
||||
The `Form` type mirrors the TOML shape one-to-one. `ValidFormTypes` is the authoritative allow-list — adding a form type means editing one map in this package.
|
||||
|
||||
### 3. HTTP handler — `internal/handler/`
|
||||
|
||||
`ContactHandler` is a single struct with four maps keyed by URL path:
|
||||
|
||||
```go
|
||||
type ContactHandler struct {
|
||||
forms map[string]*config.Form
|
||||
senders map[string]*email.FormSender
|
||||
rateLimits map[string]*rateLimiter
|
||||
stores map[string]*storage.NewsletterStore
|
||||
}
|
||||
```
|
||||
|
||||
`New(cfg)` builds all four maps in one pass. `Register(mux)` then mounts `POST <path>` and `OPTIONS <path>` per form, plus `GET /health`.
|
||||
|
||||
The per-form handler closure (`makeHandler`) runs this pipeline in order:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client
|
||||
participant Handler as ContactHandler
|
||||
participant CORS
|
||||
participant Limiter as rateLimiter
|
||||
participant Body
|
||||
participant Honey as Honeypot
|
||||
participant Validate as contactform.Validate
|
||||
participant Sender as email.FormSender
|
||||
participant Store as storage.NewsletterStore
|
||||
|
||||
Client->>Handler: POST /api/nuntius/contact
|
||||
alt preflight
|
||||
Handler->>CORS: formAllowed(origin)?
|
||||
CORS-->>Handler: yes / no
|
||||
Handler-->>Client: 204 No Content (or 403)
|
||||
else actual
|
||||
Handler->>CORS: formAllowed(origin)?
|
||||
CORS-->>Handler: yes / no
|
||||
alt origin present and not allowed
|
||||
Handler-->>Client: 403 origin_not_allowed
|
||||
end
|
||||
Handler->>Limiter: allow(ip)
|
||||
alt over limit
|
||||
Handler-->>Client: 429 rate_limited
|
||||
end
|
||||
Handler->>Body: json.Decode
|
||||
alt invalid JSON
|
||||
Handler-->>Client: 400 invalid_json
|
||||
end
|
||||
Handler->>Honey: honeypot field non-empty?
|
||||
alt bot
|
||||
Handler-->>Client: 200 ok (silent)
|
||||
end
|
||||
Handler->>Validate: contactform.Validate(req, form.Type)
|
||||
alt errors
|
||||
Handler-->>Client: 400 validation + details
|
||||
end
|
||||
Handler->>Sender: Send(req)
|
||||
alt SMTP error
|
||||
Handler-->>Client: 500 send_failed
|
||||
end
|
||||
opt form.Type == "newsletter"
|
||||
Handler->>Store: Append(Subscriber{...})
|
||||
alt write error
|
||||
Handler-->>Client: 500 storage_failed
|
||||
end
|
||||
end
|
||||
Handler-->>Client: 200 ok
|
||||
end
|
||||
```
|
||||
|
||||
Notable behaviors:
|
||||
|
||||
- **CORS is path-aware** — Each form has its own `allowed_origins` list; preflights and actual requests are checked against the form being hit, not the host.
|
||||
- **Client IP honours reverse proxies** — `clientIP` reads `X-Forwarded-For` (first hop), then `X-Real-IP`, then falls back to `RemoteAddr`. Strip the port by hand because the stdlib does not give us a `netip.AddrPort` for the remote.
|
||||
- **Rate limiting is in-process** — `rateLimiter` is a per-IP token bucket. The bucket refills at `perHour/3600` tokens per second. The map is guarded by a single `sync.Mutex`; in practice the lock is held for microseconds and the access pattern is dominated by `Allow` lookups, not insert / delete.
|
||||
- **Honeypot is a positive control** — A bot that fills the invisible field gets a 200 with no email sent and no log line. Humans are never blocked.
|
||||
- **Validation is the same function the public library uses** — `contactform.Validate` is the only validator in the codebase; the handler just calls it with `form.Type` and forwards the returned `[]FieldError` to the client.
|
||||
|
||||
### 4. Email — `internal/email/`
|
||||
|
||||
Each form gets a `FormSender` constructed once in `handler.New`. `Send(req)` is the only public method:
|
||||
|
||||
1. `smtp.PlainAuth("", user, password, host)` — the empty identity means "use the supplied user as-is", which is what every modern SMTP provider expects.
|
||||
2. `compose(from, to, req, formName, formType)` builds a `multipart/alternative` message (text/plain + text/html).
|
||||
3. `smtp.SendMail(addr, auth, from, []string{to}, msg)`.
|
||||
|
||||
`compose` picks one of four HTML templates (`contact`, `feedback`, `newsletter`, `generic`) and four matching plain-text subjects. The subject for `contact` includes the `[<service>]` tag if `req.Service` is set, so inbox filters can route by service interest.
|
||||
|
||||
Credentials are never logged — neither the password nor the full `From` / `To` headers appear in any `slog` line. The handler logs `form`, `path`, `service` (if any), and `ip` only.
|
||||
|
||||
### 5. Storage — `internal/storage/`
|
||||
|
||||
`NewsletterStore` is a thin wrapper over an append-only JSONL file:
|
||||
|
||||
| Method | Semantics |
|
||||
|---|---|
|
||||
| `Append(Subscriber)` | Creates the parent dir if missing, opens the file with `O_APPEND \| O_CREATE \| O_WRONLY`, writes one JSON line + `\n`, closes. Sets `CreatedAt = time.Now().UTC()` if zero. |
|
||||
| `Count()` | Streams the file with a `bufio.Scanner` (1 MB max line), increments on each line that unmarshals into a non-empty `Subscriber`. Malformed lines are skipped. |
|
||||
| `List()` | Same scan, returns all valid subscribers in insertion order. Reads the entire log into memory — do not call on multi-million-row files. |
|
||||
| `Path()` | Returns the configured file path. |
|
||||
|
||||
A single `sync.Mutex` guards all three methods. The store is safe for concurrent use from the handler goroutines.
|
||||
|
||||
There is **no rotation**. Add a `logrotate` unit if the file grows beyond what you want to `wc -l`.
|
||||
|
||||
## Request Flow (single form)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Browser
|
||||
participant Nginx
|
||||
participant Nuntius as nuntius
|
||||
participant SMTP
|
||||
participant Disk
|
||||
|
||||
Browser->>Nginx: POST /api/nuntius/contact
|
||||
Nginx->>Nuntius: forward (X-Forwarded-For, X-Real-IP)
|
||||
Nuntius->>Nuntius: CORS check
|
||||
Nuntius->>Nuntius: rate-limit per IP
|
||||
Nuntius->>Nuntius: parse JSON
|
||||
Nuntius->>Nuntius: honeypot check
|
||||
Nuntius->>Nuntius: contactform.Validate
|
||||
Nuntius->>SMTP: SMTP PLAIN auth + multipart message
|
||||
SMTP-->>Nuntius: 250 OK
|
||||
opt newsletter form
|
||||
Nuntius->>Disk: append one JSONL line
|
||||
end
|
||||
Nuntius-->>Nginx: 200 {"ok": true}
|
||||
Nginx-->>Browser: 200 {"ok": true}
|
||||
```
|
||||
|
||||
## Config Merging
|
||||
|
||||
There is no merging in nuntius — the TOML file is the only source of runtime configuration. The file itself goes through three stages at startup:
|
||||
|
||||
1. **Default generation** — If the file does not exist, `os.WriteFile(path, defaultConfig, 0644)` writes the embedded three-form template.
|
||||
2. **Env-var expansion** — `os.Expand` substitutes `${VAR}` and `$VAR` references **in the raw text** before parsing. The parsed struct never holds a literal like `"${NUNTIUS_SMTP_PASSWORD}"` — it holds the expanded value (or an empty string if the env var was unset).
|
||||
3. **Strict parsing + validation** — `interpres.NewDecoder(...).DisallowUnknownFields()` rejects typos; `validate()` enforces required fields, unique paths, valid form types, and applies defaults.
|
||||
|
||||
The order matters: expansion before parsing means env-var references work even for values that are not strings (e.g. an integer with a `${PORT}` reference would fail, but you can still reference strings).
|
||||
|
||||
## Error Handling
|
||||
|
||||
Errors are returned as a typed string in the JSON body, with the HTTP status code doing the heavy lifting:
|
||||
|
||||
| Status | Body shape | When |
|
||||
|---|---|---|
|
||||
| `200` | `{"ok": true}` | Submission accepted; mail sent (and subscriber persisted for `newsletter`) |
|
||||
| `200` | `{"ok": true}` | Honeypot triggered (silent accept, no mail) |
|
||||
| `400` | `{"error": "invalid_json", "message": "..."}` | Body is not valid JSON |
|
||||
| `400` | `{"error": "validation", "details": [...]}` | One or more fields are invalid |
|
||||
| `403` | `{"error": "origin_not_allowed"}` | `Origin` header is not in the form's `allowed_origins` |
|
||||
| `429` | `{"error": "rate_limited", "message": "..."}` | Token bucket empty for this IP / form |
|
||||
| `500` | `{"error": "send_failed", "message": "..."}` | SMTP round-trip failed |
|
||||
| `500` | `{"error": "storage_failed", "message": "..."}` | `newsletter` only — could not write the JSONL line |
|
||||
|
||||
The `message` field is intentionally generic; field-level details for validation errors live in `details[]`. None of the error bodies leak SMTP credentials, file paths, or stack traces.
|
||||
|
||||
## Why These Choices
|
||||
|
||||
- **stdlib only** — `net/smtp`, `net/http`, `net/mail`, `log/slog`, `bufio`, `os/signal`, `sync` plus the first-party `interpres` TOML library are all that is needed. The build is reproducible, the binary is small, and the external supply chain is empty.
|
||||
- **No global state outside the binary** — `ContactHandler` owns the maps; `slog` is the only package-level default. Everything else is constructed explicitly in `New(cfg)` so the handler is easy to instantiate from tests.
|
||||
- **Append-only JSONL** — Cheaper to implement than SQLite, easier to inspect than a custom binary format, easy to back up, and resilient to partial writes.
|
||||
- **Per-form isolation** — One form's SMTP credentials never travel through another form's code path. A misconfigured `contact` form cannot leak credentials from a `newsletter` form.
|
||||
- **Strict config parsing** — Catching `forms[0].smtp.port = "587"` (string) or `forms[0].type = "contatc"` (typo) at startup is worth the small extra error-path code.
|
||||
Reference in New Issue
Block a user