volumen/docs/ARCHITECTURE.md

Architecture

How Volumen is put together. Every node, package and arrow below exists in the source tree; nothing is aspirational.

Overview

flowchart TD
    cli["cmd/volumen, the flag dispatcher"] --> app["internal/app, server assembly and middleware chain"]
    cli --> backup["internal/backup, the tar.gz archive"]
    cli --> scheduler["internal/scheduler, the publish_at sweep"]
    app --> api["internal/httpapi, the public JSON API"]
    app --> admin["internal/admin, the server-rendered admin"]
    admin --> backup
    api --> preview["internal/preview, the signed preview link"]
    admin --> preview
    api --> store["internal/store, snapshot cache, atomic writes, revisions"]
    admin --> store
    scheduler --> store
    store --> post["internal/post, the domain object"]
    store --> disk[("content directory: posts, media, revisions")]
    post --> markdown["internal/markdown, scriptorium then bluemonday"]
    admin --> i18n["internal/i18n, the admin interface catalogue"]
    admin --> diff["internal/diff, the revision comparison"]
    app --> config["internal/config, TOML over built-in defaults"]
    admin --> fediverse["internal/fediverse, the @user@host rule"]
    config --> fediverse
    admin --> users["internal/users, internal/tokens, internal/templates"]

cmd/volumen is the only entry point. serve assembles internal/app, which builds the file-backed stores from the configuration, wires the middleware chain in front of the public API and the admin, and starts the scheduler loop when the configuration enables it; a deployment is founded through the admin itself, by the first-run wizard. export, import and publish-due reach the filesystem and the archive without starting a server. Volumen does not render the public site: the front end consumes the JSON API, and the admin is the only HTML it serves.

Packages

Package	Responsibility
config	Reads config.toml into a typed value over the built-in defaults, applies the command-line overrides and validates it; with no file the defaults move the state under the user's home. It owns the commented template (template.go) shipped as config.toml.example and nothing about the content.
store	Owns the content directory through one os.Root: reading and writing posts, the snapshot cache, per-target write locks, the revision archive and its tombstones, and the media library. It decides where a file goes and never what a post means.
imagefile	The upload rule: which extensions are accepted, the WebP, AVIF and SVG signatures, the extension a byte string is, and the pixel size the header carries (the WebP chunks, the AVIF ispe box, the SVG root's size attributes or viewBox), so the media library can show it. The store asks it what an upload is; the media route asks it what a name may be.
post	The Post domain object: frontmatter metadata, the raw Markdown body, and the lazily rendered HTML and table of contents. It derives the slug, language, date, excerpt, reading time and publication status, and it clones deeply, because the store hands one instance to several readers.
frontmatter	Parses and serialises the +++ block on an interpres Document, so a save keeps what the author wrote: the key order at every level, the comments, and whether a table is a header section or an inline table.
markdown	Owns rendering and sanitisation: scriptorium renders the body (CommonMark with the GFM extensions, footnotes and definition lists), a pre-render scan lifts $$…$$ and $…$ runs out of the source and splices the MathML back after rendering, fenced mermaid blocks become their SVG, headings gain ids and a table of contents, a figure pass wraps titled images, and the bluemonday allowlist sanitises the result. This is the only place that turns a body into HTML.
feeds	Renders RSS 2.0, Atom 1.0, JSON Feed 1.1 and the sitemap from the same posts the API serves.
payloads	Builds the API's wire shapes: site metadata, post summaries and details, tag and series listings, pagination in both modes, and the validation used by the write endpoints and the admin forms.
httpapi	Serves /api/volumen/*: reads, feeds, the sitemap, and the token-authenticated writes. It never writes a file; it calls store.
admin	Serves /admin: the first-run wizard that founds the installation, login, the dashboard, post CRUD, the editor and its preview, import and download, history, media, settings, users, tokens, webhooks, backups and self-update, with the session, role and CSRF guards.
users, tokens, templates	The file-backed stores beside users.toml: accounts with roles (the first one created by the wizard's serialised AddFirst), API token digests with scopes, and named post templates. They own their files and their atomic writes.
session	The signed session cookie: load, verify, sign, expire. The cookie also carries a fingerprint of the account's password hash, so changing a password retires every session issued before the change.
preview	The shared preview-link rule: an HMAC over the slug and an expiry stamp. The admin issues links, the API honours them, and neither implements the rule itself.
fediverse	The @user@host rule, a leaf so that the configuration, the admin account form and the post payload validation share one validator.
identifiers	The DOI and ORCID rules, a leaf like fediverse: syntax and normalisation for a DOI, shape and ISO 7064 check digit for an ORCID, and the resolver URLs both are published under.
biblio	The bibliography leaf: the refs frontmatter parsed into numbered entries, inline [n] citations linked to them, DOIs, arXiv ids and ORCIDs turned into resolver links, and the [[refs]] marker replaced by the rendered list. It imports no other domain package; the post annotates the same-instance links.
app	Assembles the server: stores, route tree, middleware chain, and the health, robots, favicon, media and 404 handlers.
web	The shared middleware (gzip, cross-origin refusal, security headers, the request logger and its id, the client address) and access to the embedded templates and assets.
i18n	The admin interface catalogue: English source strings, Czech translations, and the plural rules both languages need. The public API's messages stay English by contract.
diff	The line-based comparison behind the revision history's diff view, with a context collapse and a bounded table, so an oversized input degrades to a whole-text replacement instead of burning memory.
ratelimit	The sliding-window counter behind both the public API limit and the login limit, with a key bound enforced on every request.
webhooks	Delivers signed JSON events to the configured endpoints, with a bounded number in flight, retries, and an in-memory delivery history. Hooks come from config.toml and from the admin-managed webhooks.toml, whose changes apply through SetHooks without a restart.
backup	The one writer and reader of the tar.gz archive, shared by the CLI and the admin. It owns the layout and the containment of a restore.
audit	The append-only JSON-lines audit log.
scheduler	Publishes posts whose publish_at has arrived, from the in-app loop or the CLI, and reports each one to the webhook sink.
updater	The Gitea release check, the checksum-verified download, and the in-place replacement of the running binary.
password	scrypt hashing and verification with fixed parameters.
tomlfile	The shared atomic TOML writer (temp file, fsync, rename, mode 0600).
version	Reports the version the toolchain recorded in the build information. Nothing writes a version number.

Data flow

sequenceDiagram
    participant C as Client
    participant G as web.Gzip
    participant X as web.CrossOrigin
    participant H as web.SecurityHeaders
    participant R as apiRateLimit
    participant M as session.Middleware
    participant L as web.RequestLogger
    participant A as httpapi handleSingle
    participant S as store
    participant P as the rendering pipeline

    C->>G: GET /api/volumen/posts/hello-world
    G->>X: buffers the body when the client accepts gzip
    X->>H: refuses a state-changing request from another origin
    H->>R: baseline security headers, CSP nonce for /admin
    alt over the limit
        R-->>C: 429 with Retry-After
    else within the limit
        R->>M: X-RateLimit-Limit and X-RateLimit-Remaining set
        M->>L: the signed session cookie is loaded and verified
        L->>A: the request id is assigned and carried in the context
        A->>S: Find(slug, lang)
        S->>S: compare the path, mtime and size snapshot with the cache
        S-->>A: the post, an alias to redirect, or nothing
        A->>P: render, splice mathematics and diagrams, sanitise
        P-->>A: the HTML, cached on the post
        A-->>C: 200 with CORS, an ETag and Cache-Control
    end

The chain is built outermost first, so a request passes web.Gzip, web.CrossOrigin, web.SecurityHeaders, apiRateLimit, session.Middleware and web.RequestLogger before the route mux. The cross-origin gate is the outer one of the two write guards: it refuses a request a browser sent from another site before a handler runs, and the admin's per-session CSRF token is the inner one, which also covers a same-site request from another port. The request logger assigns a 16-character id, answers with it in X-Request-Id, puts a logger carrying it into the context so a handler's own lines share it, and writes one access line when the request finishes. The media route is split off before all of it and wrapped only in the security headers, because the session and gzip layers buffer a whole response and would hold entire images in memory.

Errors are produced close to their cause and mapped once, at the edge: the store returns an error for a failed write, the API turns it into an error envelope, and the admin renders the same message in the form it came from. A post file that cannot be parsed is never an error at the edge; the store skips it, records it, and volumen validate, volumen doctor and /healthz report it.

Admin handlers add requireLogin or requireAdmin in front of the handler and validate the CSRF token before doing any work. Write endpoints authenticate a Bearer token and check its scope. Both paths converge on the same store calls, so a post saved from the admin and one saved from the API land on disk the same way, including a rename, which moves the file and tombstones the old one.

State and lifetime

• The content directory is the only durable state. Posts, media, revisions, users.toml, templates.toml, tokens.toml, webhooks.toml and the audit log all live on disk; a restart loses only the in-memory caches, the sitemap memo and the webhook delivery history.
• One process, one content directory. Every cache and limiter is in-process, so two servers must never share a content directory. The lockers are for concurrent requests inside one process, not for two writers on one tree.
• Cached posts are shared. store.All and store.Find return the same *post.Post to several readers, so a writer clones first (Post.Clone deep-copies the metadata). The rendered HTML is cached on the instance with a sync.Once, which is safe for concurrent readers by construction.
• Locks. The store holds one mutex over its cache and snapshot, a reference-counted mutex per write target, and its own guard for that map. The users, tokens, templates and audit stores each hold one mutex; the rate limiters hold one each and bound their key sets.
• Sessions live in the cookie, signed with the secret from [admin].session_key, or from the secret.key the server generated beside the users file when the config leaves the key empty, and bound to the account's password hash by a fingerprint, so a password change ends every session issued before it while the device that made the change re-signs itself. A request that carries a valid cookie is authenticated without server state, so signing out expires the browser's copy rather than revoking the value, and rotating the key is what ends every session at once.
• Long-lived goroutines are the scheduler loop and the HTTP server; both are joined on shutdown, which drains in-flight requests for up to 15 seconds after SIGINT or SIGTERM. Webhook deliveries and the release check are bounded, detached, and die with the process.
• Post files are written atomically: a temp file in the target directory, fsync, rename, directory fsync. The previous version is archived before the rename, and a delete is a move into that archive, so both are reversible.
• The content directory is reached through an os.Root. Every read, write and delete inside it goes through the handle, which refuses a path that would escape the tree through .. or a symlink, so confinement is a property of the store rather than a check each caller has to remember.

Dependencies

The direct requires in go.mod are the whole list, and each is there because the standard library does not do the job:

• sourcedock.dev/petrbalvin/scriptorium renders the Markdown body, the TeX mathematics and the Mermaid diagrams, deterministically and on the standard library alone.
• bluemonday sanitises the rendered HTML against an allowlist. It is the reason a post body can be treated as untrusted even though its author is authenticated.
• sourcedock.dev/petrbalvin/interpres/v2 parses TOML. It keeps a real date a date, rather than turning date = 2026-01-15 into a locale-dependent guess, and its Document keeps key order and comments so a save round-trips the author's own frontmatter.
• golang.org/x/crypto provides scrypt, which the standard library does not ship, for password hashing.
• golang.org/x/text provides the NFKC normalisation applied to passwords before hashing, so a password typed with a different Unicode form still verifies.

Everything else is the standard library: net/http for the server and its routing, html/template and embed for the admin, archive/tar and compress/gzip for the archive, encoding/json/v2 for the API, and log/slog for diagnostics.