petrbalvin/volumen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Initial commit
Test / test (push) Successful in 7m5s
Details
Release / gates (push) Successful in 7m28s
Details
Release / build (amd64, freebsd) (push) Successful in 2m52s
Details
Release / build (amd64, linux) (push) Successful in 2m46s
Details
Release / build (arm64, freebsd) (push) Successful in 2m22s
Details
Release / build (arm64, linux) (push) Successful in 2m38s
Details
Release / build (loong64, linux) (push) Successful in 2m7s
Details
Release / build (riscv64, linux) (push) Successful in 2m17s
Details
Release / release (push) Successful in 1m0s
Details

Browse Source 
Assisted-by: GLM 5.3
This commit is contained in:
petrbalvin
2026-09-29 10:03:32 +02:00
commit f8ed33df83
206 changed files with 44165 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/API.md
+670
View File
@@ -0,0 +1,670 @@
# API
Volumen serves a JSON API under `/api/volumen`; the admin interface at
`/admin` is server-rendered HTML for humans and is not part of this API.
## HTTP API
Base URL: `{base_url}/api/volumen`, where `base_url` is the `[site].base_url`
setting ([CONFIGURATION.md](CONFIGURATION.md)). Every path below is relative
to that prefix.
Reads need no credential. A write carries `Authorization: Bearer <token>`,
where the token is minted in the admin at **Settings, API tokens** and shown
once, at creation. Two scopes exist: `write` for `POST` and `PUT`, and
`delete` for `DELETE`. A token created with no scope selected is unrestricted,
and a scope list that names no recognised scope is refused rather than turned
into an unrestricted token. There is no read scope, because every read
endpoint is public, so a token can only widen access to the write and delete
operations.
A `HEAD` request is answered as the `GET` of the same path: same status, same
headers, no body.
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/api/volumen/site` | Site metadata |
| `GET` | `/api/volumen/posts` | Paginated, filterable post list |
| `GET` | `/api/volumen/posts/batch` | Several post details in one request |
| `GET` | `/api/volumen/posts/{slug}` | One post with body and rendered HTML |
| `POST` | `/api/volumen/posts` | Create a post (scope `write`) |
| `PUT` | `/api/volumen/posts/{slug}` | Update a post (scope `write`) |
| `DELETE` | `/api/volumen/posts/{slug}` | Delete a post (scope `delete`) |
| `GET` | `/api/volumen/tags` | Tag cloud with counts |
| `GET` | `/api/volumen/tags/{tag}` | Posts carrying one tag |
| `GET` | `/api/volumen/tags/{tag}/feed.xml` | RSS 2.0 feed for one tag |
| `GET` | `/api/volumen/tags/{tag}/feed.atom` | Atom 1.0 feed for one tag |
| `GET` | `/api/volumen/tags/{tag}/feed.json` | JSON Feed 1.1 for one tag |
| `GET` | `/api/volumen/series` | Series list with post counts |
| `GET` | `/api/volumen/series/{name}` | Posts of one series, in reading order |
| `GET` | `/api/volumen/series/{name}/feed.xml` | RSS 2.0 feed for one series |
| `GET` | `/api/volumen/series/{name}/feed.atom` | Atom 1.0 feed for one series |
| `GET` | `/api/volumen/series/{name}/feed.json` | JSON Feed 1.1 for one series |
| `GET` | `/api/volumen/feed.xml` | RSS 2.0 feed |
| `GET` | `/api/volumen/feed.atom` | Atom 1.0 feed |
| `GET` | `/api/volumen/feed.json` | JSON Feed 1.1 document |
| `GET` | `/api/volumen/sitemap.xml` | XML sitemap |
| `OPTIONS` | `/api/volumen/{rest...}` | CORS preflight for the whole subtree |
### `GET /api/volumen/site`
Site metadata, taken from the configuration. Every value is echoed as
configured and every one of them is a string, so `fediverse_creator` is `""`
when the key is unset rather than `null`.
```sh
curl -s 'https://lab.example.com/api/volumen/site'
```
```json
{
"title": "Research Notes",
"description": "Powered by Volumen.",
"base_url": "https://lab.example",
"language": "en",
"author": "Anonymous",
"fediverse_creator": ""
}
```
The response carries an `ETag`; send it back in `If-None-Match` to receive
`304 Not Modified` with no body. The post list, the post batch, the tag cloud,
a tag's post list and a single post carry one as well. The feeds, the sitemap
and the other endpoints do not.
### `GET /api/volumen/posts`
| Parameter | Type | Default | Effect |
|---|---|---|---|
| `page` | integer | `1` | Page number, `1` to `1000000`. Ignored when `cursor` is set |
| `limit` | integer | `20` | Page size, `1` to `100` |
| `lang` | string | unset | Keep posts whose language is this value, plus posts flagged `all_langs` |
| `tag` | string | unset | Keep posts carrying this tag |
| `q` | string | unset | Case-insensitive search over the title, the tags, the excerpt and the body. Matching posts are ordered by relevance: a title hit outranks a tag hit, an excerpt hit and a body hit, and a title that starts with the query leads its class; equal scores keep the date order |
| `cursor` | string | unset | Slug to start after; switches the response to the cursor shape |
A malformed `page` or `limit` is rejected with `422` and the `validation`
envelope. An unknown slug in `cursor` yields a page with no posts and a `null`
cursor rather than silently rewinding to the first page, so a paging client
never receives posts it already holds.
```sh
curl -s 'https://lab.example.com/api/volumen/posts?limit=20'
```
```json
{
"page_size": 20,
"total": 2,
"posts": [
{
"slug": "alpha",
"title": "Alpha",
"excerpt": "Alpha body with a [link](https://example.com).",
"date": "2026-08-18",
"lang": "cs",
"tags": ["go", "research"],
"author": "Petr",
"fediverse_creator": "@petr@social",
"cover": "/media/c.webp",
"cover_alt": "alt",
"cover_caption": "caption",
"reading_time": 1,
"translations": { "en": "alpha-en" },
"series": "Series",
"series_order": 1,
"url": "/api/volumen/posts/alpha"
},
{
"slug": "beta",
"title": "Beta",
"excerpt": "Beta body.",
"date": "2026-07-01",
"lang": "en",
"tags": ["go"],
"reading_time": 1,
"url": "/api/volumen/posts/beta"
}
],
"page": 1,
"has_next": false,
"has_prev": false
}
```
Without a cursor the response carries `page`, `has_next` and `has_prev`. With
a cursor it carries `next_cursor` instead: the slug of the last post on the
page, or `null` when the list is exhausted. When two posts share a slug (a
post and its translation may) the cursor is `slug.lang`, so the next page
resumes after the exact post rather than after the first variant that
matches. Either way `total` counts every post the filter matched, before
pagination, and `page_size` is the limit the server applied.
```json
{
"page_size": 1,
"total": 2,
"posts": [
{
"slug": "beta",
"title": "Beta",
"excerpt": "Beta body.",
"date": "2026-07-01",
"lang": "en",
"tags": ["go"],
"reading_time": 1,
"url": "/api/volumen/posts/beta"
}
],
"next_cursor": null
}
```
#### Fields
Used by the list endpoints, the tag and series lists, the series detail and
the write responses. The shape is `payloads.Summary`, built by
`payloads.BuildSummary`; the struct tags decide which fields are omitted.
| Field | Type | Presence | Notes |
|---|---|---|---|
| `slug` | string | omitted when empty | Stable post identifier |
| `title` | string | omitted when empty | Post title |
| `excerpt` | string | always present | The frontmatter excerpt, else the first non-heading body paragraph cut to 200 characters, with an ellipsis when anything was dropped |
| `date` | string | omitted when empty | Publication date, `YYYY-MM-DD` |
| `lang` | string | omitted when empty | Post language |
| `tags` | array of strings | omitted when empty | Post tags |
| `author` | string | omitted when empty | Post author from the frontmatter; the site author is not substituted into the payload |
| `fediverse_creator` | string | omitted when empty | `@user@host`, per-post override of the site default |
| `doi` | string | omitted when empty | Digital Object Identifier, bare `10.…/…` form; a stored `doi.org` URL or `doi:` prefix is normalised for display, and a value that fails the syntax rule passes through untouched. Set in the frontmatter or the admin editor, not by the write endpoints |
| `orcid` | string | omitted when empty | The author's ORCID iD, `0000-0000-0000-000X`, check digit verified when the value is written from the admin editor; upper-cased for display when valid, the raw text otherwise. Set in the frontmatter or the admin editor, not by the write endpoints |
| `cover` | string | omitted when empty | Cover image path or URL, typically `/media/...` |
| `cover_alt` | string | omitted when empty | Alt text for the cover |
| `cover_caption` | string | omitted when empty | Caption for the cover |
| `reading_time` | integer | always present | Minutes, `ceil(words / 200)`, at least 1 |
| `translations` | object | omitted when empty | Maps a language code to the slug of a sibling translation |
| `series` | string | omitted when empty | Series name |
| `series_order` | integer | omitted when the frontmatter has none | Position within the series |
| `url` | string | always present | API detail URL, `/api/volumen/posts/{slug}` |
### `GET /api/volumen/posts/batch`
Returns the detail of several posts in one response. `slugs` is a
comma-separated list; entries are trimmed, empty ones dropped, and the list is
capped at 100 slugs. A slug that names no published post is skipped, so the
`posts` array may be shorter than the request and the endpoint never answers
`404`. Without `slugs` the `posts` array is empty.
```sh
curl -s 'https://lab.example.com/api/volumen/posts/batch?slugs=alpha,beta'
```
The entry below is one detail response, printed in full:
```json
{
"posts": [
{
"slug": "alpha",
"title": "Alpha",
"excerpt": "Alpha body with a [link](https://example.com).",
"date": "2026-08-18",
"lang": "cs",
"tags": ["go", "research"],
"author": "Petr",
"fediverse_creator": "@petr@social",
"cover": "/media/c.webp",
"cover_alt": "alt",
"cover_caption": "caption",
"reading_time": 1,
"translations": { "en": "alpha-en" },
"series": "Series",
"series_order": 1,
"url": "/api/volumen/posts/alpha",
"body": "Alpha **body** with a [link](https://example.com).\n",
"html": "<p>Alpha <strong>body</strong> with a <a rel=\"noopener noreferrer\" href=\"https://example.com\">link</a>.</p>\n",
"toc": "<div class=\"toc\">\n<ul></ul>\n</div>\n",
"meta": {
"url": "https://lab.example/alpha",
"json_ld": "{\"@context\":\"https://schema.org\",\"@type\":\"Article\",\"author\":{\"@type\":\"Person\",\"name\":\"Petr\"},\"creator\":{\"@type\":\"Person\",\"name\":\"@petr@social\"},\"dateModified\":\"2026-08-18\",\"datePublished\":\"2026-08-18\",\"description\":\"Alpha body with a [link](https://example.com).\",\"headline\":\"Alpha\",\"image\":[\"/media/c.webp\"],\"inLanguage\":\"cs\",\"keywords\":[\"go\",\"research\"],\"mainEntityOfPage\":{\"@id\":\"https://lab.example/alpha\",\"@type\":\"WebPage\"},\"url\":\"https://lab.example/alpha\"}",
"og": {
"article:author": "Petr",
"article:published_time": "2026-08-18",
"article:tag": ["go", "research"],
"og:description": "Alpha body with a [link](https://example.com).",
"og:image": "/media/c.webp",
"og:locale": "cs",
"og:title": "Alpha",
"og:type": "article",
"og:url": "https://lab.example/alpha"
},
"twitter": {
"twitter:card": "summary_large_image",
"twitter:creator": "@petr@social",
"twitter:description": "Alpha body with a [link](https://example.com).",
"twitter:image": "/media/c.webp",
"twitter:title": "Alpha"
}
}
}
]
}
```
The JSON here is reformatted for the page; the response shape is pinned by
the fixtures under
[`internal/httpapi/testdata/contract/`](../internal/httpapi/testdata/contract/),
which carry a differently configured deployment's values.
### `GET /api/volumen/posts/{slug}`
| Parameter | Type | Default | Effect |
|---|---|---|---|
| `lang` | string | unset | Select the language variant; a post flagged `all_langs` matches every value |
| `preview_token` | string | unset | Signed preview credential for a draft or a scheduled post |
The response is a post detail. A slug that matches an alias is answered with
`301 Moved Permanently` and a `Location` header naming the canonical post. A
draft and a scheduled post are distinguishable on purpose: the error code says
which of the two withheld the post, because the client already knows the slug
and the two states need different handling.
The response carries an `ETag` naming the post's state: send it back in
`If-None-Match` for a `304`, or in `If-Match` on a write to refuse overwriting
a change you never saw.
| Status | Meaning |
|---|---|
| `200` | The post detail |
| `301` | The slug is an alias; `Location` names the canonical slug |
| `304` | The request carried `If-None-Match` matching the current `ETag` |
| `404` | `{"error": "not_found"}`, or `{"error": "draft"}` / `{"error": "scheduled"}` for a withheld post with no valid preview token |
| `500` | `{"error": "render_failed"}` when the Markdown pipeline fails |
A preview link is signed as an HMAC over the slug and an expiry stamp, keyed
with `[admin].session_key`, and stays valid for seven days. Without a session
key no link is issued at all: the admin route `GET
/admin/posts/{slug}/preview-link` then answers `409` with the code
`no_session_key`. A preview response carries `Cache-Control: no-store`: it
holds unpublished content, and a shared cache must not keep it.
#### Fields
Returned by `GET /api/volumen/posts/{slug}` and by the batch endpoint. The
shape is `payloads.Detail`: the summary above, plus:
| Field | Type | Notes |
|---|---|---|
| `body` | string | Raw Markdown source |
| `html` | string | Rendered, sanitised HTML |
| `toc` | string | Sanitised table of contents |
| `meta` | object | SEO and discovery metadata, computed per request from the post and never persisted |
| `references` | array of objects | The structured bibliography from the post's `refs` frontmatter: each entry has `num`, optional `authors` (`{name, orcid}`), `title`, `venue`, `year`, `volume`, `pages`, `doi`, `arxiv`, `url`, or the verbatim `raw` text. When the entry's DOI is published by another post of the same instance, it also carries `internal`: the API address of that post, so a consumer can keep the reader on site. Omitted when the post cites nothing |
| `fields` | object | The frontmatter keys the engine does not consume itself, the author's own: `colour: "#0f0"` lands here as `{"colour": "#0f0"}`. Nested tables, arrays and TOML date-times pass through (dates as their canonical strings). Omitted when every key is a known one |
`html` is rendered by scriptorium (CommonMark, GFM, footnotes, definition lists) with
`$$…$$` and `$…$` mathematics converted to MathML Core and fenced mermaid blocks
(flowcharts and sequence diagrams) drawn as inline SVG, and it is
sanitised by bluemonday against a narrow allowlist: headings keep their `id`
attributes, fenced code blocks carry `class="language-..."`, an image whose
title is set becomes a `figure` with a `figcaption`, a post with `refs` gets a
numbered `<section class="refs" id="references">` spliced at its `[[refs]]`
marker, or appended at the end of the body when it carries none, with every
inline `[n]` citation linked to it. A reference whose DOI
is published by another post of the same instance links to that post's API
address instead of the external resolver, and the same address appears as
the `url` of its JSON-LD citation next to the canonical identifier; a post
citing its own DOI keeps the resolver link. Every link in the
body and in the table of contents gains `rel="noopener noreferrer"`. The
allowlist admits `http`, `https` and `mailto` URLs plus relative ones, so a
body cannot inject a script, a style or an event handler. A mermaid diagram
the renderer does not carry stays a fenced code block, and a mathematics
construct outside the mappable surface stays visible as its verbatim source
in the place it was written.
`toc` is the wrapper `<div class="toc">` with a nested list of heading links.
The wrapper is always present, even when the body has no headings, in which
case it holds an empty list:
```json
"toc": "<div class=\"toc\">\n<ul></ul>\n</div>\n"
```
An empty body has no table of contents at all and yields an empty `toc`
string.
`meta` carries:
| Field | Type | Notes |
|---|---|---|
| `url` | string | Canonical post URL: `base_url` plus the slug, or the API path when no `base_url` is set |
| `json_ld` | string | A Schema.org `Article` document, serialised as a JSON string |
| `og` | object | Open Graph fields, keyed `og:type`, `og:title`, `og:description`, `og:url`, `og:locale`, `article:published_time`, plus `og:image`, `article:tag` and `article:author` when the post carries them |
| `twitter` | object | Card fields, keyed `twitter:card`, `twitter:title`, `twitter:description`, plus `twitter:image` and `twitter:creator` when the post carries them |
### `POST /api/volumen/posts`
Creates a post. The body is a JSON object; fields not listed are ignored. The
response is `201 Created` with the post summary as its body, no `Location`
header, and an `ETag` naming the new state for a later `If-Match` write.
| Field | Type | Effect |
|---|---|---|
| `title` | string | Post title |
| `slug` | string | Post slug; required, and unique |
| `lang` | string | Language code |
| `author` | string | Author name |
| `fediverse_creator` | string | `@user@host` handle |
| `excerpt` | string | Summary text, overrides the derived excerpt |
| `body` | string | Markdown source, at most 1 MiB |
| `tags` | array of strings, or a comma-separated string | Post tags |
| `cover` | string | Cover image path or URL |
| `cover_alt` | string | Cover alt text |
| `cover_caption` | string | Cover caption |
| `series` | string | Series name |
| `series_order` | integer | Position within the series |
| `date` | string | Publication date, `YYYY-MM-DD` |
| `publish_at` | string | Scheduled publication date, `YYYY-MM-DD` |
| `draft` | boolean | Withhold the post |
| `all_langs` | boolean | Serve the post for every requested language |
An empty string or `null` clears an optional field. A value of the wrong type
is rejected with `400` rather than coerced. Validation failures answer
`{"error": "validation", "message": "..."}` with messages such as
`Slug is required.`, `Invalid slug.`, `A post with that slug already exists.`
and `Body must be at most 1048576 bytes.`
```sh
curl -s -X POST 'https://lab.example.com/api/volumen/posts' \
-H 'Authorization: Bearer vol_...' \
-H 'Content-Type: application/json' \
-d '{"slug":"hello","title":"Hello","body":"Hi.","tags":["meta"]}'
```
### `PUT /api/volumen/posts/{slug}`
Updates the post named by the path. Fields omitted from the body keep their
stored value, so the request is a merge rather than a replacement. The
response is `200 OK` with the post summary.
A `slug` in the body that differs from the path renames the post: the file is
written under the new slug and the old file is soft-deleted, so the rename
stays undoable.
The request may carry `If-Match` with the `ETag` a `GET` of the post served:
when it no longer names the stored state, the write is refused with `412`
instead of overwriting a change the client never saw. `If-Match: *` demands
only that the post exists. A write without the header stays unconditional.
The `200` response carries the new state's `ETag`, so edits can chain without
re-reading. The precondition is checked against the post the path names, the
one a `GET` without `lang` serves.
| Status | Meaning |
|---|---|
| `200` | The post summary, with the new `ETag` |
| `400` | `{"error": "invalid_json"}` or the `validation` envelope |
| `401` | `{"error": "unauthorized"}` |
| `403` | `{"error": "forbidden", "message": "Token lacks 'write' scope"}` |
| `404` | `{"error": "not_found"}` |
| `412` | `{"error": "precondition_failed"}` when `If-Match` names an older state |
| `500` | `{"error": "save_failed"}` when the file cannot be written, or `{"error": "render_failed"}` when an `If-Match` precondition has to render a stored post that no longer renders |
### `DELETE /api/volumen/posts/{slug}`
Deletes the post. The response is `204 No Content` with no body. The post is
soft-deleted as a tombstone, so it can be restored from the admin while the
tombstone survives.
An `If-Match` header carrying the `ETag` a `GET` of the post served refuses
the delete with `412` when it names an older state; `*` and a missing header
are unconditional.
| Status | Meaning |
|---|---|
| `204` | Deleted |
| `401` | `{"error": "unauthorized"}` |
| `403` | `{"error": "forbidden", "message": "Token lacks 'delete' scope"}` |
| `404` | `{"error": "not_found"}` |
| `412` | `{"error": "precondition_failed"}` when `If-Match` names an older state |
| `500` | `{"error": "delete_failed"}`, or `{"error": "render_failed"}` when an `If-Match` precondition has to render a stored post that no longer renders |
### `GET /api/volumen/tags`
```json
{
"tags": [
{ "name": "go", "count": 2 },
{ "name": "research", "count": 1 }
]
}
```
Tags are ordered by count, most used first, then by name. Only published posts
are counted. The response carries an `ETag`.
### `GET /api/volumen/tags/{tag}`
A page of the posts carrying one tag, in the shape of
[`GET /api/volumen/posts`](#get-apivolumenposts) with the same `page`,
`limit`, `lang` and `cursor` parameters. A tag that names no published post
answers `404` with `{"error": "not_found"}`.
### `GET /api/volumen/series`
```json
{
"series": [{ "name": "Series", "count": 1 }]
}
```
Ordered by count, most posts first, then by name.
### `GET /api/volumen/series/{name}`
```json
{
"name": "Series",
"count": 1,
"posts": [
{
"slug": "alpha",
"title": "Alpha",
"excerpt": "Alpha body with a [link](https://example.com).",
"date": "2026-08-18",
"lang": "cs",
"tags": ["go", "research"],
"author": "Petr",
"reading_time": 1,
"series": "Series",
"series_order": 1,
"url": "/api/volumen/posts/alpha"
}
]
}
```
The posts are in reading order: by `series_order` first, and a post without
one after every ordered post, then by date and slug. A name that matches no
series answers `404` with `{"error": "not_found"}`.
### Feeds
| Path | Document |
|---|---|
| `/api/volumen/feed.xml` | RSS 2.0, `application/rss+xml` |
| `/api/volumen/feed.atom` | Atom 1.0, `application/atom+xml` |
| `/api/volumen/feed.json` | JSON Feed 1.1, `application/json` |
| `/api/volumen/tags/{tag}/feed.xml`, `/api/volumen/tags/{tag}/feed.atom`, `/api/volumen/tags/{tag}/feed.json` | The same three formats for one tag |
| `/api/volumen/series/{name}/feed.xml`, `/api/volumen/series/{name}/feed.atom`, `/api/volumen/series/{name}/feed.json` | The same three formats for one series |
A feed carries at most 20 posts: the newest 20 for the site feed and a tag
feed, and the first 20 of the series, in its reading order, for a series feed.
Every document names its own address: RSS carries `<atom:link rel="self">`,
Atom carries `<link rel="self">`, and JSON Feed carries `feed_url`, each with
the path that was actually requested, so a tag feed or a series feed reports
its own URL rather than the site feed's. A tag or series feed answers `404`
with `{"error": "not_found"}` when nothing matches.
### `GET /api/volumen/sitemap.xml`
Every published post, truncated to the newest 50 000 (the ceiling the
sitemaps.org protocol sets for one document), with `<lastmod>` from the file's
modification time and, for a post whose file cannot be read, from its date.
The content type is `application/xml`.
### `OPTIONS /api/volumen/{rest...}`
The CORS preflight for the whole API subtree. It advertises
`GET, POST, PUT, DELETE, OPTIONS` with `Content-Type, Authorization`, sets
`Access-Control-Max-Age: 600`, and answers `200` with an empty
`text/plain; charset=utf-8` body. The write methods are listed on purpose: a
browser preflight for a cross-origin `POST` fails when the answer names only
`GET`. The answer carries the write CORS set rather than the read one, so the
allowed origin follows the rule the Headers section states for a
token-authenticated write.
### Flow
The token-authenticated write flow, from the request to the stored file:
```mermaid
sequenceDiagram
participant Client
participant API
participant Tokens as Token store
participant Posts as Content directory
Client->>API: POST /api/volumen/posts with Authorization: Bearer
API->>Tokens: Authenticate the raw token
Tokens->>Tokens: compare the SHA-256 digest
Tokens-->>API: token record with its scopes
API->>API: require the write scope
API->>Posts: Save the post, archiving the previous version
Posts-->>API: the saved path
API-->>Client: 201 Created with the post summary
```
At each step, a failure surfaces in the response: a missing or unknown token
as `401 unauthorized`, a token without the scope as `403 forbidden`, a body
that is not a JSON object as `400 invalid_json`, and a rejected field as `400
validation`. When the post is stored, the `post.created` event fires and every
webhook subscribed to it receives the post summary.
## Notes
The Go structs and their JSON tags are the authority on the response shapes,
and the committed fixtures under
[`internal/httpapi/testdata/contract/`](../internal/httpapi/testdata/contract/)
pin them: a change to a body or a status code fails the contract test until
the fixture is regenerated with `go test ./internal/httpapi -update-contract`
and the change is recorded in the changelog. `go doc ./internal/payloads`
covers the exported types. This file explains what the surface is for; it does
not repeat the struct definitions, because a copy of a signature is a future
lie.
### Errors
Every failure uses one envelope: an `error` code, optionally with a
human-readable `message` and extra fields.
```json
{ "error": "not_found" }
```
```json
{ "error": "validation", "message": "Invalid slug.", "field": "slug" }
```
| Code | Status | Extra fields | Meaning |
|---|---|---|---|
| `not_found` | 404 | | No such post, tag, series, or media file |
| `draft` | 404 | | The post exists but is a draft |
| `scheduled` | 404 | | The post exists but is scheduled for the future |
| `validation` | 400, 422 | `message`, and `field` for a query parameter | The payload or query is invalid |
| `invalid_json` | 400 | | The request body is not a JSON object |
| `payload_too_large` | 413 | | The request body exceeds 10 MiB |
| `precondition_failed` | 412 | | The request's `If-Match` names an older state than the stored one |
| `unauthorized` | 401 | | Missing or invalid bearer token |
| `forbidden` | 403 | `message` | The token lacks the required scope |
| `rate_limited` | 429 | `retry_after` | Too many requests |
| `render_failed` | 500 | | The Markdown pipeline failed |
| `save_failed` | 500 | | The post could not be written to disk |
| `delete_failed` | 500 | | The post could not be deleted |
A `401` carries `WWW-Authenticate: Bearer`. Failures written by the API carry
`Cache-Control: no-store`, so a shared cache never keeps one; the `429` is
written by the rate limiter before the handler and carries the rate-limit
headers instead.
The router answers for itself outside the routes above: a path under
`/api/volumen/` that matches no route, or a method a route does not serve,
gets `405 Method Not Allowed` with an `Allow` header listing the methods the
path does serve, and a plain-text body rather than the JSON envelope.
### Headers and caching
- JSON responses use `Content-Type: application/json`. The XML documents carry
`application/rss+xml`, `application/atom+xml` and `application/xml`.
- A read answers with the permissive set:
`Access-Control-Allow-Origin: *`, `Access-Control-Allow-Methods: GET, POST,
PUT, DELETE, OPTIONS` and `Access-Control-Allow-Headers: Content-Type,
Authorization`. A token-authenticated write echoes the configured `base_url`
as the allowed origin when the request carries an `Origin` header, and falls
back to `*` when no base URL is configured or the request carries no `Origin`
header, so a browser can read a
cross-origin write from the site itself and not from anywhere else. The
router's `405` and the rate limiter's `429` are written outside a handler and
carry only what their own sections note.
- Successful reads carry `Cache-Control: public, max-age=60,
stale-while-revalidate=21600`. Write responses carry `no-store`.
- Bodies of 500 bytes or more are gzip-compressed when the client sends
`Accept-Encoding: gzip`, and such a response carries `Vary:
Accept-Encoding` so a shared cache keys on it.
- Dates are ISO 8601 (`YYYY-MM-DD`).
- Lists are newest first by post date. Drafts and scheduled posts never appear.
- Empty optional fields are omitted rather than sent as `null`; a client must
treat a missing key as "not set". Two fields of a post summary are the
exception and are always present: `excerpt` and `reading_time`. The site
payload is different in kind: it echoes the configured values, and every one
of them is a string, so its `fediverse_creator` is `""` when the key is
unset.
- Bodies are written by the Go 1.27 `encoding/json/v2` encoder with
deterministic member order and no HTML escaping, so `<`, `>` and `&` appear
as themselves and the same content produces the same bytes. That is what
makes the `ETag` round trip above reliable. A request body that names the
same member twice is rejected as `invalid_json` rather than silently taking
one of the two values.
- Malformed query parameters are rejected with `422` and the validation
envelope.
### Rate limiting
With `[api].rate_limit` above zero, every `/api/volumen/*` response carries
`X-RateLimit-Limit` and `X-RateLimit-Remaining`, counted per client address in
a sliding window of `[api].rate_limit_window` seconds. A request over the
budget is answered `429`:
```json
{ "error": "rate_limited", "retry_after": 42 }
```
with a `Retry-After` header carrying the same number of seconds. The limiter
covers the API prefix only: `/media/*`, `/admin/*` and `/healthz` are not
counted. The client address is the last entry of `X-Forwarded-For` when
`[server].trust_proxy` is set and the connection comes from a peer listed in
`[server].trusted_proxies`, and the connection address otherwise. The refusal
carries the read CORS set (`GET, OPTIONS` with `Content-Type`) rather than the
wide one, and no cache header.
### Operational routes
Outside the API prefix, the application serves a few operational routes:
| Route | Behaviour |
|---|---|
| `GET /healthz` | `{"status": "ok", "checks": {...}}` with `Cache-Control: no-store`. The status is `degraded` and the code `503` when the content directory is missing, a post file cannot be parsed, the users, tokens or templates file cannot be read, or the filesystem has under 100 MB free. `content_dir`, `users_file` and `disk` are always present; `content_files`, `tokens_file` and `templates_file` appear only when something is wrong with them. Not rate limited |
| `GET /robots.txt` | Allows all crawlers and points at the sitemap: `Sitemap: {base_url}/api/volumen/sitemap.xml` |
| `GET /sitemap.xml` | `301` redirect to `/api/volumen/sitemap.xml` |
| `GET /favicon.ico` | The bundled SVG icon, `Cache-Control: public, max-age=86400` |
| `GET /media/{path}` | An uploaded image from `<content_dir>/media/`. The file name is a UUID with a `.webp`, `.avif` or `.svg` extension, assigned when the upload is stored, so no uploader chooses a name; only those three extensions resolve, and anything else, including a missing file, answers `404` with `{"error": "not_found"}`. The type is set from the extension rather than sniffed, so a file whose bytes do not match is still served as an image and never as a document; an SVG, the one accepted image that is also a document, is additionally served with a sandboxing `Content-Security-Policy`, so opened at its own URL it runs as no one. Responses carry `Cache-Control: public, max-age=604800`. They are served outside the gzip and session layers, so a large image is never buffered in memory |
| `GET /` | `303` redirect to `/admin/` |
| anything else outside the API prefix | `404` with `{"error": "not_found"}` |
docs/ARCHITECTURE.md
+198
View File
@@ -0,0 +1,198 @@
# Architecture
How Volumen is put together. Every node, package and arrow below exists in the
source tree; nothing is aspirational.
## Overview
```mermaid
flowchart TD
cli["cmd/volumen, the flag dispatcher"] --> app["internal/app, server assembly and middleware chain"]
cli --> backup["internal/backup, the tar.gz archive"]
cli --> scheduler["internal/scheduler, the publish_at sweep"]
app --> api["internal/httpapi, the public JSON API"]
app --> admin["internal/admin, the server-rendered admin"]
admin --> backup
api --> preview["internal/preview, the signed preview link"]
admin --> preview
api --> store["internal/store, snapshot cache, atomic writes, revisions"]
admin --> store
scheduler --> store
store --> post["internal/post, the domain object"]
store --> disk[("content directory: posts, media, revisions")]
post --> markdown["internal/markdown, scriptorium then bluemonday"]
admin --> i18n["internal/i18n, the admin interface catalogue"]
admin --> diff["internal/diff, the revision comparison"]
app --> config["internal/config, TOML over built-in defaults"]
admin --> fediverse["internal/fediverse, the @user@host rule"]
config --> fediverse
admin --> users["internal/users, internal/tokens, internal/templates"]
```
`cmd/volumen` is the only entry point. `serve` assembles `internal/app`, which
builds the file-backed stores from the configuration, wires the middleware
chain in front of the public API and the admin, and starts the scheduler loop
when the configuration enables it; a deployment is founded through the admin
itself, by the first-run wizard. `export`, `import` and
`publish-due` reach the filesystem and the archive without starting a server.
Volumen does not render the public site: the front end consumes the JSON API,
and the admin is the only HTML it serves.
## Packages
| Package | Responsibility |
|---|---|
| `config` | Reads `config.toml` into a typed value over the built-in defaults, applies the command-line overrides and validates it; with no file the defaults move the state under the user's home. It owns the commented template (`template.go`) shipped as `config.toml.example` and nothing about the content. |
| `store` | Owns the content directory through one `os.Root`: reading and writing posts, the snapshot cache, per-target write locks, the revision archive and its tombstones, and the media library. It decides where a file goes and never what a post means. |
| `imagefile` | The upload rule: which extensions are accepted, the WebP, AVIF and SVG signatures, the extension a byte string is, and the pixel size the header carries (the WebP chunks, the AVIF `ispe` box, the SVG root's size attributes or `viewBox`), so the media library can show it. The store asks it what an upload is; the media route asks it what a name may be. |
| `post` | The `Post` domain object: frontmatter metadata, the raw Markdown body, and the lazily rendered HTML and table of contents. It derives the slug, language, date, excerpt, reading time and publication status, and it clones deeply, because the store hands one instance to several readers. |
| `frontmatter` | Parses and serialises the `+++` block on an interpres `Document`, so a save keeps what the author wrote: the key order at every level, the comments, and whether a table is a header section or an inline table. |
| `markdown` | Owns rendering and sanitisation: scriptorium renders the body (CommonMark with the GFM extensions, footnotes and definition lists), a pre-render scan lifts `$$…$$` and `$…$` runs out of the source and splices the MathML back after rendering, fenced mermaid blocks become their SVG, headings gain ids and a table of contents, a figure pass wraps titled images, and the bluemonday allowlist sanitises the result. This is the only place that turns a body into HTML. |
| `feeds` | Renders RSS 2.0, Atom 1.0, JSON Feed 1.1 and the sitemap from the same posts the API serves. |
| `payloads` | Builds the API's wire shapes: site metadata, post summaries and details, tag and series listings, pagination in both modes, and the validation used by the write endpoints and the admin forms. |
| `httpapi` | Serves `/api/volumen/*`: reads, feeds, the sitemap, and the token-authenticated writes. It never writes a file; it calls `store`. |
| `admin` | Serves `/admin`: the first-run wizard that founds the installation, login, the dashboard, post CRUD, the editor and its preview, import and download, history, media, settings, users, tokens, webhooks, backups and self-update, with the session, role and CSRF guards. |
| `users`, `tokens`, `templates` | The file-backed stores beside `users.toml`: accounts with roles (the first one created by the wizard's serialised `AddFirst`), API token digests with scopes, and named post templates. They own their files and their atomic writes. |
| `session` | The signed session cookie: load, verify, sign, expire. The cookie also carries a fingerprint of the account's password hash, so changing a password retires every session issued before the change. |
| `preview` | The shared preview-link rule: an HMAC over the slug and an expiry stamp. The admin issues links, the API honours them, and neither implements the rule itself. |
| `fediverse` | The `@user@host` rule, a leaf so that the configuration, the admin account form and the post payload validation share one validator. |
| `identifiers` | The DOI and ORCID rules, a leaf like `fediverse`: syntax and normalisation for a DOI, shape and ISO 7064 check digit for an ORCID, and the resolver URLs both are published under. |
| `biblio` | The bibliography leaf: the `refs` frontmatter parsed into numbered entries, inline `[n]` citations linked to them, DOIs, arXiv ids and ORCIDs turned into resolver links, and the `[[refs]]` marker replaced by the rendered list. It imports no other domain package; the post annotates the same-instance links. |
| `app` | Assembles the server: stores, route tree, middleware chain, and the health, robots, favicon, media and 404 handlers. |
| `web` | The shared middleware (gzip, cross-origin refusal, security headers, the request logger and its id, the client address) and access to the embedded templates and assets. |
| `i18n` | The admin interface catalogue: English source strings, Czech translations, and the plural rules both languages need. The public API's messages stay English by contract. |
| `diff` | The line-based comparison behind the revision history's diff view, with a context collapse and a bounded table, so an oversized input degrades to a whole-text replacement instead of burning memory. |
| `ratelimit` | The sliding-window counter behind both the public API limit and the login limit, with a key bound enforced on every request. |
| `webhooks` | Delivers signed JSON events to the configured endpoints, with a bounded number in flight, retries, and an in-memory delivery history. Hooks come from `config.toml` and from the admin-managed `webhooks.toml`, whose changes apply through `SetHooks` without a restart. |
| `backup` | The one writer and reader of the tar.gz archive, shared by the CLI and the admin. It owns the layout and the containment of a restore. |
| `audit` | The append-only JSON-lines audit log. |
| `scheduler` | Publishes posts whose `publish_at` has arrived, from the in-app loop or the CLI, and reports each one to the webhook sink. |
| `updater` | The Gitea release check, the checksum-verified download, and the in-place replacement of the running binary. |
| `password` | scrypt hashing and verification with fixed parameters. |
| `tomlfile` | The shared atomic TOML writer (temp file, `fsync`, rename, mode `0600`). |
| `version` | Reports the version the toolchain recorded in the build information. Nothing writes a version number. |
## Data flow
```mermaid
sequenceDiagram
participant C as Client
participant G as web.Gzip
participant X as web.CrossOrigin
participant H as web.SecurityHeaders
participant R as apiRateLimit
participant M as session.Middleware
participant L as web.RequestLogger
participant A as httpapi handleSingle
participant S as store
participant P as the rendering pipeline
C->>G: GET /api/volumen/posts/hello-world
G->>X: buffers the body when the client accepts gzip
X->>H: refuses a state-changing request from another origin
H->>R: baseline security headers, CSP nonce for /admin
alt over the limit
R-->>C: 429 with Retry-After
else within the limit
R->>M: X-RateLimit-Limit and X-RateLimit-Remaining set
M->>L: the signed session cookie is loaded and verified
L->>A: the request id is assigned and carried in the context
A->>S: Find(slug, lang)
S->>S: compare the path, mtime and size snapshot with the cache
S-->>A: the post, an alias to redirect, or nothing
A->>P: render, splice mathematics and diagrams, sanitise
P-->>A: the HTML, cached on the post
A-->>C: 200 with CORS, an ETag and Cache-Control
end
```
The chain is built outermost first, so a request passes `web.Gzip`,
`web.CrossOrigin`, `web.SecurityHeaders`, `apiRateLimit`, `session.Middleware`
and `web.RequestLogger` before the route mux. The cross-origin gate is the
outer one of the two write guards: it refuses a request a browser sent from
another site before a handler runs, and the admin's per-session CSRF token is
the inner one, which also covers a same-site request from another port. The
request logger assigns a 16-character id, answers with it in `X-Request-Id`,
puts a logger carrying it into the context so a handler's own lines share it,
and writes one access line when the request finishes. The media route is split
off before all of it and wrapped only in the security headers, because the
session and gzip layers buffer a whole response and would hold entire images in
memory.
Errors are produced close to their cause and mapped once, at the edge: the
store returns an error for a failed write, the API turns it into an error
envelope, and the admin renders the same message in the form it came from. A
post file that cannot be parsed is never an error at the edge; the store skips
it, records it, and `volumen validate`, `volumen doctor` and `/healthz` report
it.
Admin handlers add `requireLogin` or `requireAdmin` in front of the handler
and validate the CSRF token before doing any work. Write endpoints
authenticate a `Bearer` token and check its scope. Both paths converge on the
same store calls, so a post saved from the admin and one saved from the API
land on disk the same way, including a rename, which moves the file and
tombstones the old one.
## State and lifetime
- **The content directory is the only durable state.** Posts, media,
revisions, `users.toml`, `templates.toml`, `tokens.toml`, `webhooks.toml`
and the audit log all live on disk; a restart loses only the in-memory
caches, the sitemap memo and the webhook delivery history.
- **One process, one content directory.** Every cache and limiter is
in-process, so two servers must never share a content directory. The lockers
are for concurrent requests inside one process, not for two writers on one
tree.
- **Cached posts are shared.** `store.All` and `store.Find` return the same
`*post.Post` to several readers, so a writer clones first (`Post.Clone`
deep-copies the metadata). The rendered HTML is cached on the instance with
a `sync.Once`, which is safe for concurrent readers by construction.
- **Locks.** The store holds one mutex over its cache and snapshot, a
reference-counted mutex per write target, and its own guard for that map.
The users, tokens, templates and audit stores each hold one mutex; the rate
limiters hold one each and bound their key sets.
- **Sessions live in the cookie**, signed with the secret from
`[admin].session_key`, or from the `secret.key` the server generated beside
the users file when the config leaves the key empty, and
bound to the account's password hash by a fingerprint, so a password change
ends every session issued before it while the device that made the change
re-signs itself. A request that carries a valid cookie is authenticated
without server state, so signing out expires the browser's copy rather than
revoking the value, and rotating the key is what ends every session at once.
- **Long-lived goroutines** are the scheduler loop and the HTTP server; both
are joined on shutdown, which drains in-flight requests for up to 15 seconds
after `SIGINT` or `SIGTERM`. Webhook deliveries and the release check are
bounded, detached, and die with the process.
- **Post files are written atomically**: a temp file in the target directory,
`fsync`, rename, directory `fsync`. The previous version is archived before
the rename, and a delete is a move into that archive, so both are reversible.
- **The content directory is reached through an `os.Root`.** Every read, write
and delete inside it goes through the handle, which refuses a path that
would escape the tree through `..` or a symlink, so confinement is a
property of the store rather than a check each caller has to remember.
## Dependencies
The direct requires in `go.mod` are the whole list, and each is there because
the standard library does not do the job:
- `sourcedock.dev/petrbalvin/scriptorium` renders the Markdown body, the TeX
mathematics and the Mermaid diagrams, deterministically and on the standard
library alone.
- `bluemonday` sanitises the
rendered HTML against an allowlist. It is the reason a post body can be
treated as untrusted even though its author is authenticated.
- `sourcedock.dev/petrbalvin/interpres/v2` parses TOML. It keeps a real date a
date, rather than turning `date = 2026-01-15` into a locale-dependent guess,
and its `Document` keeps key order and comments so a save round-trips the
author's own frontmatter.
- `golang.org/x/crypto` provides scrypt, which the standard library does not
ship, for password hashing.
- `golang.org/x/text` provides the NFKC normalisation applied to passwords
before hashing, so a password typed with a different Unicode form still
verifies.
Everything else is the standard library: `net/http` for the server and its
routing, `html/template` and `embed` for the admin, `archive/tar` and
`compress/gzip` for the archive, `encoding/json/v2` for the API, and
`log/slog` for diagnostics.
docs/BENCHMARKING.md
+92
View File
@@ -0,0 +1,92 @@
# Benchmarking
How **Volumen** is measured. Every number a document, a README or a changelog
quotes comes from here and nowhere else.
## The method
The benchmarks live next to the code they measure. The tree carries exactly
two, and nothing else is measured:
| Benchmark | What it measures |
|---|---|
| `BenchmarkRender` | the Markdown pipeline, from source to sanitised HTML and a table of contents, for a medium body and a large one |
| `BenchmarkServer` | the wired server over real HTTP: a post list, a single post, the tag cloud, a tag feed and the sitemap, against a corpus of five hundred posts |
`BenchmarkRender` in `internal/markdown/markdown_test.go` has two
sub-benchmarks and calls `SetBytes`, so a result reads as input bytes per
second: the medium input is 2400 bytes of Markdown, the large one eighteen
copies of it, 43200 bytes.
`BenchmarkServer` in `internal/app/bench_test.go` builds a server over a
temporary content directory and drives it through an `httptest` server, so it
covers the whole chain rather than one function: routing, the middleware, the
session layer, the store, the payload builders and the renderer. It is the
workload a release is judged on, and the one a profile is recorded from.
- The machine is the development workstation, idle: AMD Ryzen AI MAX+ PRO 395
with Radeon 8060S, 16 cores and 32 threads, 117 GiB of memory, Fedora Linux
44 (`linux/amd64`). A loaded box times whatever else is running, and the
fastest sample can land on the wrong function.
- The toolchain is `go1.27.1 linux/amd64`. Benchmarks run through `go test`,
which builds the package's test binary; the command build's `-trimpath` and
`-buildvcs=true` are not part of a benchmark's build.
- Comparisons run inside one process. A loaded machine and separate processes
of identical binaries differ by more than the effects being measured, so
A/B runs alternate the two sides rather than run one after the other, and
the counts are compared through their medians, with the allocations and the
bytes per operation alongside the times. Differences within a few percent
of the spread between runs are noise; only a difference beyond that is a
result.
- When timing is hopeless, the allocation and byte counts are the result.
- A profile says where the time goes, and it is only read from an idle
machine. Record one with `-cpuprofile` on `BenchmarkServer`, then `go tool
pprof -top` it.
- Profile-guided optimisation: a profile is committed at
`cmd/volumen/default.pgo`, and the toolchain consumes it automatically when
it builds the command (measured: a `-pgo=off` build and a default build of
the same tree produce different binaries). A benchmark's test binary never
sees it, because the profile belongs to the main package, so the early A/B
run of `BenchmarkServer` compared two plain builds and could not have shown
a difference; its numbers stand as the plain build's, and they understate
the shipped binary. The comparison measured on 2026-09-25 against the two
real builds (five hundred posts, the six JSON and feed endpoints
interleaved, rate limiting off, both arms served the same twelve thousand
six hundred requests) puts the profile's effect beyond the noise: server
CPU over the warm mix is about six percent lower, the per-request median
about thirty-two percent lower on `site`, twenty-three percent on the post
list and fifteen percent on a single post, while the tag feed, the sitemap
and the cold store scan are unchanged within the noise. The profile
predates the search-relevance and custom-fields changes to the hot path.
## Running
```sh
just bench
```
The recipe runs the whole module with five counts:
```sh
go test -run '^$' -bench=. -benchmem -count=5 ./...
```
`-benchmem` is not optional: allocations per operation are part of the
result. A first look at one target, before the full battery is worth the
time:
```sh
go test -run '^$' -bench 'BenchmarkServer' -benchmem -benchtime=1x -count=1 ./internal/app
```
The full battery runs once, deliberately, on an idle machine. A benchmark
command is capped at about two minutes per round; longer sweeps are split.
A server benchmark also spends time in the kernel, so read the allocation
numbers alongside the time: a change that halves allocations and leaves the
time flat has moved the cost to the filesystem.
## Reports
The repository stores no benchmark reports. A performance claim in
`CHANGELOG.md` is measured with the method above on the change that makes
it, on the named machine, and the number travels with the claim.
docs/CLI.md
+273
View File
@@ -0,0 +1,273 @@
# Command line
The reference below is taken from the program's own `--help`. If the two
disagree, the program is right and this file is a defect. The same reference
ships as a manual page, [man/volumen.1](../man/volumen.1), and the page moves
in the same commit as the flags it documents.
## Synopsis
```sh
volumen <command> [options]
```
## Commands
| Command | Purpose |
|---|---|
| `serve` | Start the publishing server |
| `status` | Show installation status |
| `doctor` | Check installation health |
| `check-update` | Compare with the latest Gitea release |
| `export` | Export posts, media, and users to a tar.gz |
| `import` | Import a backup archive |
| `publish-due` | Publish scheduled posts whose date arrived |
| `validate` | Validate the content directory |
| `version` | Show version |
## `volumen serve`
```text
volumen serve [--config <path>] [--content <dir>] [--host <addr>] [--port <n>]
```
| Flag | Default | Meaning |
|------|---------|---------|
| `--config` | `/etc/volumen/config.toml`, then `~/.config/volumen/config.toml` | Path to `config.toml` |
| `--content` | from config | Override the posts directory |
| `--host` | from config (`::`) | Override the bind address |
| `--port` | from config (`9091`) | Override the port |
A `--port` outside `1..65535` is rejected rather than ignored, and a positional
argument is an error.
Starts the HTTP server: the public API under `/api/volumen`, the admin under
`/admin`, uploaded media under `/media`, plus `/healthz`, `/robots.txt`,
`/sitemap.xml` and `/favicon.ico`. The configuration is validated at startup;
invalid values abort with exit code `1`, before the socket is bound.
With no `--config` the server reads `/etc/volumen/config.toml` if it exists,
then `~/.config/volumen/config.toml`, and when neither exists it runs on the
built-in defaults. Those defaults put the state under the user's home
(`~/.local/share/volumen`, honouring `XDG_DATA_HOME`), so a plain
`volumen serve` on a fresh machine works without root and without any
configuration file. On the first start the account file is empty and
`/admin` shows the first-run wizard: it creates the administrator account,
the interface language and the colour scheme, and signs the operator in. The
server keeps the session secret it generates in `secret.key` beside the
account file; `[admin].session_key` overrides it.
The server sets a 10 second read-header timeout, a 60 second read timeout, a
120 second write timeout and a 120 second idle timeout, and bounds a request
line and its headers at 1 MiB. `SIGINT` and `SIGTERM` drain in-flight requests
for up to 15 seconds and then exit `0`.
When `[scheduler].enabled = true`, an internal goroutine publishes the due
posts once at start-up and then every `[scheduler].interval` seconds. A
background release check fills the admin update banner without blocking
requests. Logging goes to stderr, either as text or as JSON lines when
`[server].log_format = "json"`; the server's own protocol errors go there too,
through `log/slog`, rather than to a bare stderr line. Every request carries a
16-character id: it is answered in `X-Request-Id`, it is attached to every line
the handlers write for that request, and the request's own access line records
the method, the path, the status and the duration under it.
## `volumen status`
```text
volumen status [--config <path>] [--data <dir>] [--users-file <path>] [--json]
```
Reports whether the config exists, parses and validates, the number of posts
(with the draft count), and the number of users; with no accounts yet the user
count carries the hint `open /admin to run the setup wizard`. `--json` prints
`{"ok": true, "checks": {…}}`, with a `config_validate` entry naming the first
invalid value, a `posts_unreadable` entry when a post file cannot be parsed, and
`users: "unreadable"` when the accounts file cannot be read. Exit code `1` when
the config is missing, unparseable or invalid, when content cannot be parsed, or
when the users file cannot be read.
Read-only: it inspects the content directory without creating or changing
anything, so a mistyped `content_dir` is reported rather than turned into an
empty tree.
## `volumen doctor`
```text
volumen doctor [--config <path>] [--json]
```
Runs the health checks: config presence and validation, whether every post file
can be parsed, whether every post renders, whether the users file can be read,
whether the installation has its first account, and whether any stored scrypt
parameters are below the policy floor. The
checks carry two levels: a missing or broken config, an unreadable content
directory and an unreadable users file are `fail` and exit code `1`; posts
that fail to render, an installation still waiting for the first-run wizard
and weak stored hashes are `warn`, reported without
changing the exit code. `--json` prints
`{"ok": bool, "checks": [{"name", "status", "detail"}]}`.
## `volumen check-update`
```text
volumen check-update [--json]
```
Compares the running version with the latest release published on the Gitea
instance (`https://sourcedock.dev/petrbalvin/volumen/releases`).
| Exit code | Meaning |
|-----------|---------|
| `0` | Up to date (or the local build is newer) |
| `1` | A newer release is available, or the release check failed (offline, timeout, malformed response) |
| `2` | The arguments were wrong |
`--json` prints `{"current": …, "latest": …, "available": bool}` and keeps the
same exit-code contract, so it works as a CI or monitoring probe.
## `volumen export`
```text
volumen export [--config <path>] [--out <archive>]
```
Writes a `.tar.gz` containing `posts/` (including `.revisions/` and `media/`),
`users.toml`, `templates.toml` and `tokens.toml`. Default output:
`volumen-backup.tar.gz`, written mode `0600`.
The config file is deliberately not included: it may hold the session signing
key override, and `volumen import` ignores it anyway. A file that exists but cannot be read
aborts the export rather than producing an archive that looks complete: the
archive is written to a temporary file and renamed into place, so a failure
leaves the previous backup untouched. Exit code `1` on I/O errors.
## `volumen import`
```text
volumen import [--config <path>] <archive>
```
Restores an archive produced by `volumen export` or the admin Backup panel
into the configured locations. An archive entry named `users.toml`,
`templates.toml` or `tokens.toml` is written to the path the deployment
configures for that file, which may be a different name; the post files under
`posts/` are written into the content directory.
Extraction is confined to the content directory, so an entry that tries to
escape it is refused, and only post files, revision archives and images with an
allowed extension are written: the media directory is served from a public
route, so an archive can never plant a document there. The decompressed total
is bounded. Exit code `1` on a malformed archive or one that holds no file the
layout recognises.
## `volumen publish-due`
```text
volumen publish-due [--config <path>] [--dry-run] [--json]
```
Publishes every post whose `publish_at` date is today or earlier: removes the
`publish_at` key and defaults `date` to it when the post has no date. This is
the cron and systemd-timer counterpart to the in-app `[scheduler]`.
- `--dry-run` lists the due slugs without touching files.
- `--json` prints `{"published": […], "failed": n}` (or
`{"due": […], "dry_run": true}`).
- The configured webhooks receive `post.published` for every post this
publishes, the same event the admin delivers.
- Exit code `1` when a post could not be written, so a timer notices.
## `volumen validate`
```text
volumen validate [--config <path>] [--json]
```
Scans the content directory and reports: files that cannot be parsed, posts
that fail to render, duplicate slugs, slug values that do not match the slug
format, missing titles, a `publish_at` that is not a date (which withholds the
post), and alias conflicts (an alias used twice or shadowing a live slug).
Prints `content OK` when clean; exit code `1` with a problem list otherwise.
`--json` prints `{"problems": [{"slug", "path", "error"}]}`, where `path` names
the file when there is no slug to name.
A file that cannot be parsed is invisible to every other command, which is why
it is named here first.
## `volumen version`
```text
volumen version
```
Prints `volumen <version>`: the release the toolchain recorded in the
binary's build information. A release binary reports its tag (`v1.0.0`); a
build from a plain checkout reports a pseudo-version naming the commit; a
build outside version control reports `(devel)`, and a dirty tree appends
`+dirty`. Nothing injects the version, so the reported value cannot go stale.
## Global flags
| Flag | Effect |
|---|---|
| `-h`, `--help`, `help` | prints the usage block and exits `0` |
| `-v`, `--version` | prints `volumen <version>` and exits `0` |
| `-h` on a subcommand | prints that subcommand's flags and exits `0`; `version` prints the version instead |
An unknown command and an unknown flag exit `2`. Every subcommand rejects a
stray positional argument with exit code `2`; `import` is the one command whose
positional argument (`<archive>`) is required.
## Exit codes
| Code | Meaning |
|---|---|
| `0` | success |
| `1` | a failure the program detected: an unreadable config, a failed write, a content directory with a problem, a release check that could not run |
| `2` | the arguments were wrong |
`check-update` is the exception that proves the rule: it exits `1` both when a
newer release exists and when the release check failed, so a monitoring probe
reads its `--json` output rather than its status alone.
## Examples
Serve a checkout against its own content, with the scheduler on:
```sh
volumen serve --config ./config.toml --content ./posts
```
Start a fresh per-user installation and open the wizard:
```sh
volumen serve
```
The server comes up on `http://localhost:9091`; `/admin` shows the
first-run wizard, which creates the administrator account and signs the
operator in. The state lives under `~/.local/share/volumen`, and the
session secret in `secret.key` beside it.
Publish what a timer missed, then check the content directory:
```sh
volumen publish-due --dry-run
volumen publish-due
volumen validate
```
Back up a deployment and restore it into a second one:
```sh
volumen export --config /etc/volumen/config.toml --out /backup/volumen.tar.gz
volumen import --config /srv/staging/config.toml /backup/volumen.tar.gz
```
Find out why a fresh install will not start:
```sh
volumen doctor --config /etc/volumen/config.toml
volumen status --config /etc/volumen/config.toml --json
```
docs/CONFIGURATION.md
+272
View File
@@ -0,0 +1,272 @@
# Configuration
Volumen reads its configuration from one TOML file:
`/etc/volumen/config.toml` for a system install,
`~/.config/volumen/config.toml` for a per-user install, with the file that
exists tried first when no `--config` is given. With no file anywhere the
server runs on the built-in defaults, which put the state under
`~/.local/share/volumen`. No environment variable is read (the XDG ones only
locate those paths). Flags passed to `volumen serve` override the file, as
described under [Precedence](#precedence).
## File
The file is copied from the commented template (`internal/config/template.go`),
which is committed as `config.toml.example` at the repository root. Every key
the loader accepts is listed there with its
meaning, so an operator sees the whole surface and edits what this deployment
needs rather than discovering keys from a bare table dump. The copy lives at
the config path with mode `600`, because it is deployment state.
```toml
# volumen configuration.
#
# Every key this file accepts is listed here. Copy it to
# /etc/volumen/config.toml (or ~/.config/volumen/config.toml for a
# per-user installation), then edit.
#
# Keys that belong to no table come first, because a key written below a
# [table] header belongs to that table.
# Directory of the Markdown posts (.md with TOML frontmatter).
content_dir = "/var/lib/volumen/posts"
# File holding the admin accounts (managed from the admin Settings page).
users_file = "/var/lib/volumen/users.toml"
# How many previous versions of each post to keep in .revisions/
# (0 keeps none, which also makes deleting a post permanent).
revision_limit = 10
# Where the audit log is appended, or "" to disable auditing. Records
# who changed what, and when, in JSON lines.
audit_log = ""
[server]
# Address to bind, as an IP address: "::" is every interface, "::1" is
# loopback only, which is what a reverse proxy needs.
host = "::"
port = 9091
# Environment label: "development" or "production". It decides the
# startup safety checks (session key length, cookie flags, password
# policy).
env = "development"
# Set true ONLY when a trusted reverse proxy terminates TLS in front of
# volumen. Client addresses are then taken from X-Forwarded-For and
# cookies are marked Secure.
trust_proxy = false
# Addresses whose X-Forwarded-For may be believed, as addresses or CIDR
# prefixes. An empty list never reads the header and always uses the
# connection address; list the proxy so its clients each rate-limit
# under their own address.
trusted_proxies = []
# Set true in production to force the Secure flag on session cookies.
cookie_secure = false
# Log output format: "text" (human readable) or "json" (structured).
log_format = "text"
[site]
title = "Volumen"
description = "Powered by Volumen."
# Absolute URL of the public site, without a trailing slash.
base_url = "https://example.com"
language = "en"
author = "Anonymous"
# Fediverse handle surfaced as the author in feeds and meta tags.
# Leave empty to disable.
fediverse_creator = ""
[admin]
# Secret that signs session cookies (at least 64 bytes in production).
# Leave empty: the server generates one and keeps it in secret.key next
# to users.toml. A value here overrides that file.
session_key = ""
# Session lifetime in seconds (24 hours by default).
session_ttl = 86400
# Minimum password length enforced when a password is set in the admin UI.
min_password_length = 10
# Maximum password length, to bound the scrypt work.
max_password_length = 1024
# Maximum upload size in bytes (10 MB by default).
max_upload_bytes = 10485760
[api]
# Public API rate limit: requests allowed per window per client address.
# 0 disables rate limiting.
rate_limit = 60
# Rate-limit window length in seconds.
rate_limit_window = 60
# Scheduled publishing, for a post whose frontmatter carries publish_at.
# [scheduler]
# enabled = false
# interval = 300
# Outgoing webhooks: POST a signed JSON payload on post changes so a
# front-end can rebuild its cache or static pages. Repeat the block for
# more endpoints; events may be omitted to receive every event.
# [[webhooks]]
# url = "https://example.com/hooks/rebuild"
# secret = "a-long-random-string" # HMAC-SHA256 signing key
# events = ["post.created", "post.updated", "post.deleted", "post.published"]
# enabled = true
```
The template lists the four keys that belong to no table first, before any
header, because TOML puts a key written below a `[table]` header inside that
table. The loader refuses such a file rather than falling back to the default:
a `content_dir` written under `[server]` stops the start with a message naming
the fix, so a misplaced key is never silently ignored.
Five state files live next to `users_file`, in the same directory, and are not
read from the configuration file itself: `users.toml` holds the admin
accounts, started by the first-run wizard; `secret.key` holds the session
secret the server generated on first start (unless `[admin].session_key`
overrides it); `templates.toml` the `[[templates]]` array of post templates offered
in the admin "New post" form, each with `name`, `title`, `slug`, `tags`,
`body`, and an optional `[templates.fields]` table of editor inputs to
pre-fill (`author`, `lang`, `doi`, `orcid`, `series`, `series_order`,
`excerpt`, `cover`, …); `tokens.toml` the API token records, each with `name`, `token_hash`
(the SHA-256 digest, never the raw token), `created`, `last_used` and `scopes`;
and `webhooks.toml` the admin-managed webhook endpoints, of the same shape as
`[[webhooks]]` below. The admin rewrites each atomically at mode `600`, so
editing one by hand while the server runs is not advised. A token record
without a `scopes` list is unrestricted; a token created with a scope list
that names no recognised scope is refused rather than turned into an
unrestricted one; the scopes are `write` and `delete`
([`internal/tokens/tokens.go`](../internal/tokens/tokens.go)). A `webhooks.toml`
that cannot be parsed is logged and ignored, and the config-declared hooks
keep working.
## Keys
| Key | Type | Default | Effect |
|---|---|---|---|
| `content_dir` | string | `"/var/lib/volumen/posts"` | Directory of `.md` files with `+++` TOML frontmatter. Archived revisions live under `<content_dir>/.revisions/`, uploaded media under `<content_dir>/media/`. The parent directory must be writable, checked at startup with a write probe |
| `users_file` | string | `"/var/lib/volumen/users.toml"` | File holding the admin accounts. Its parent directory must be writable. `templates.toml`, `tokens.toml` and `webhooks.toml` are created next to it |
| `revision_limit` | integer | `10` | Archived versions kept per post under `<content_dir>/.revisions/`. `0` disables archiving: a save then keeps no previous version and a delete is permanent |
| `audit_log` | string | `""` | Path of the JSON-lines audit log, or empty to disable auditing. Each line is one JSON object carrying `ts` (RFC 3339, UTC), `action` and `user`, and, where the action knows them, `resource`, `detail` and `ip`. The file is created at mode `600`; a write failure is logged and never fatal |
| `server.host` | string | `"::"` | Bind address, written as an IP address rather than a name: the address is parsed, not resolved, so `"localhost"` is refused. `"::"` listens on every interface with IPv4 dual-stack; `"::1"` or `"127.0.0.1"` binds loopback only, for a reverse proxy in front |
| `server.port` | integer | `9091` | TCP port to bind. `1` to `65535` |
| `server.env` | string | `"development"` | `"development"` or `"production"`. The label decides one thing beyond its own validation: in production the session-key rules are fatal, as the `admin.session_key` row and [Validation](#validation) describe |
| `server.trust_proxy` | boolean | `false` | Take the client address from `X-Forwarded-For` instead of the connection, and mark session cookies `Secure`. The header is read only when the peer address is inside `server.trusted_proxies`, and the address taken is its last entry, the one the proxy appends when it forwards a request; the entries to its left are client-supplied and can be forged to rotate the rate-limit key, so the proxy must append the connecting address rather than pass the header through. Set it only behind a trusted reverse proxy that terminates TLS, and outside production the start logs a warning saying so |
| `server.trusted_proxies` | array of strings | `[]` | Addresses or CIDR prefixes whose `X-Forwarded-For` may be believed. An empty list never reads the header and always uses the connection address, so every client behind the proxy shares one rate-limit budget; a loopback deployment behind nginx lists `["::1", "127.0.0.1"]`. Each entry must parse as an address or a prefix. The address taken is always the header's last entry, the one the trusted proxy appended; with two chained proxies in front of the listener that entry is the front proxy's address, so all of its clients then share one rate-limit bucket, and the deployment must either let only the immediate proxy append the header or size the limit for the aggregate |
| `server.cookie_secure` | boolean | `false` | Mark session cookies `Secure` so a browser sends them over HTTPS only. Cookies also carry the flag when `server.trust_proxy` is set, and either setting makes responses carry `Strict-Transport-Security` |
| `server.log_format` | string | `"text"` | `"text"` for human-readable lines or `"json"` for one structured object per line through `log/slog`. Applied once at startup by `volumen serve` |
| `site.title` | string | `"Volumen"` | Site title, served by `/api/volumen/site` and used in the feeds |
| `site.description` | string | `"Powered by Volumen."` | Site description, used in the RSS channel, the Atom subtitle, the JSON Feed and `/api/volumen/site` |
| `site.base_url` | string | `"https://example.com"` | Canonical site URL, without a trailing slash. Must be an absolute URL. Builds every absolute link: post permalinks in the feeds, `feed_url`, the sitemap, preview links and the `Sitemap:` line in `robots.txt` |
| `site.language` | string | `"en"` | Default language, inherited by a post whose frontmatter and directory carry none. Emitted as `<language>` in RSS and `language` in the JSON Feed. Seeds the admin interface language for the login screen until an account picks its own |
| `site.author` | string | `"Anonymous"` | Site author, served by `/api/volumen/site`. It is not substituted into a post's own `author` field |
| `site.fediverse_creator` | string | `""` | Optional site-wide handle in `@user@host` form, validated against that pattern when set. Served by `/api/volumen/site`, used as the author of a JSON Feed item whose post has none, and offered as the default in the admin post form. A per-post value takes precedence |
| `admin.session_key` | string | `""` | Secret that signs session cookies and preview links. Leave it empty: the server generates a 64-character hex secret on first start and keeps it in `secret.key` beside the users file, so sessions survive restarts without the operator doing anything. A value here overrides that file and must be at least 64 bytes in production; a shorter configured value is refused at startup there, and tolerated in development, where an empty or short key means an ephemeral secret |
| `admin.session_ttl` | integer | `86400` | Session lifetime in seconds, used as the cookie `max-age` and enforced when the cookie is loaded. `1` to `31536000` (24 hours by default, one year at most) |
| `admin.min_password_length` | integer | `10` | Shortest password the admin accepts when an account is created or a password is changed. At least `1` and at most `admin.max_password_length` |
| `admin.max_password_length` | integer | `1024` | Longest password the admin accepts. Bounds the scrypt work, because unbounded input would be a denial-of-service vector. `1` to `1024`; a larger value is refused at startup rather than turning into a failed password change later |
| `admin.max_upload_bytes` | integer | `10485760` | Largest accepted upload in bytes: a media upload (`POST /admin/uploads`, refused with `413` and the code `too_large`), a post import and a profile photo, which report the size in the form instead. `1` to `1073741824` (10 MiB by default, 1 GiB at most) |
| `api.rate_limit` | integer | `60` | Requests allowed per window per client address across `/api/volumen/*`, counted in a sliding window. `0` disables rate limiting: the middleware is not installed and no `X-RateLimit-*` header is sent. Zero or greater. See the [API documentation](API.md#rate-limiting) for the headers and the `429` body |
| `api.rate_limit_window` | integer | `60` | Window length in seconds for the limit above. `1` to `86400` while rate limiting is enabled; a value outside that range is refused at startup |
| `scheduler.enabled` | boolean | `false` | Start the in-process publish loop, which is the alternative to running `volumen publish-due` from cron: with the scheduler enabled, `volumen serve` publishes due posts itself, once at startup and then on the interval. Read once, at startup |
| `scheduler.interval` | integer | `300` | Seconds between runs. At least `1` when the scheduler is enabled |
| `[[webhooks]].url` | string | unset | Endpoint URL, required, absolute `http` or `https`; an entry without it stops the start |
| `[[webhooks]].secret` | string | `""` | HMAC-SHA256 signing key. When set, each request carries `X-Volumen-Signature: sha256=<hex digest of the raw body>` |
| `[[webhooks]].events` | array of strings | `[]` | Events to receive: `post.created`, `post.updated`, `post.deleted`, `post.published`. Empty or absent receives every event |
| `[[webhooks]].enabled` | boolean | `true` | `false` keeps the entry but skips delivery |
Both scheduled-publishing mechanisms do the same work: a post whose
`publish_at` date is today or earlier loses that key and gains a `date` when
it carried none, and each published post fires the `post.published` webhook.
See [DEPLOYMENT.md](DEPLOYMENT.md#scheduled-publishing).
The optional `[[webhooks]]` array registers one outgoing webhook per entry.
When a post is created, updated, deleted or published, Volumen POSTs a signed
JSON body to every matching endpoint. Every request also carries
`X-Volumen-Event` (the event name), `X-Volumen-Delivery` (a unique delivery
id) and `User-Agent: volumen/<version>`. The body carries `event`,
`timestamp`, `version` and, for a post event, a `post` object holding the
post summary. Delivery runs in the background with up to three attempts. The
most recent deliveries are listed in the admin at **Settings, Webhooks**,
where a **Send test** button fires a `ping` event. Endpoints added in the
admin are not written into this file: they live in `webhooks.toml` beside the
users file, and a change applies there without a restart; the config-declared
entries are read-only in the admin and both sets deliver.
## Precedence
Sources, strongest first: the flags of `volumen serve`, then the configuration
file, then the built-in defaults. There is no environment variable and no
second file.
| Flag | Effect |
|---|---|
| `--config PATH` | Selects the file to read. Without it the server tries `/etc/volumen/config.toml`, then `~/.config/volumen/config.toml`, and uses the built-in defaults when neither exists |
| `--content DIR` | Overrides `content_dir` |
| `--host ADDR` | Overrides `server.host` |
| `--port N` | Overrides `server.port`, and must be `1` to `65535` |
```sh
volumen serve --config /etc/volumen/config.toml \
--content /var/lib/volumen/posts \
--host 127.0.0.1 \
--port 9000
```
A key absent from the file takes its default, and a key the loader does not
know is ignored, so a file left over from an older release still starts. A
missing file is not an error either: the defaults are used and one line is
logged saying so.
## Validation
`Config.Validate()` runs at startup, before the server binds.
The file is decoded into typed values, and a key whose TOML type does not match
its field is an error rather than a silent fallback to the default, because a
typo that quietly disables a setting is worse than a refusal to start. The
decoder lives in [`internal/config/config.go`](../internal/config/config.go)
and covers every key in the table above. A key the decoder does not know is
ignored, so a file written for another release still loads, and a root key
written below a table header is refused with the fix in the message, as the
[File](#file) section describes. The message names the key and the type it
found:
```text
volumen serve: parse config /etc/volumen/config.toml: interpres: server.port: cannot assign string to int
```
The remaining rules are value rules:
| Key | Rule |
|---|---|
| `server.host` | Non-empty, and an IP address |
| `server.port` | `1` to `65535` |
| `server.env` | `development` or `production` |
| `server.log_format` | `text` or `json` |
| `server.trusted_proxies` | Each entry an address or a CIDR prefix |
| `site.base_url` | Non-empty, and an absolute URL with a scheme and a host |
| `site.fediverse_creator` | `@user@host` when set |
| `admin.session_ttl` | `1` to `31536000` |
| `admin.min_password_length` | At least `1`, and at most `admin.max_password_length` |
| `admin.max_password_length` | At most `1024` |
| `admin.max_upload_bytes` | `1` to `1073741824` |
| `admin.session_key` | At least 64 bytes when `env = "production"` |
| `revision_limit` | Zero or greater |
| `api.rate_limit` | Zero or greater |
| `api.rate_limit_window` | `1` to `86400` while rate limiting is enabled |
| `scheduler.interval` | At least `1` while the scheduler is enabled |
| `[[webhooks]].url` | Absolute `http` or `https` URL |
| `content_dir` | The parent directory must be creatable and writable |
| `users_file` | The parent directory must be creatable and writable |
A failure stops the process with the message on standard error and exit code
`1`, as the example above shows. `volumen doctor --config PATH` reports the
same check without starting the server:
```text
config ok /etc/volumen/config.toml
config-validate ok
posts-readable ok
posts-render ok
users-file ok
password-hashes ok
```
docs/DEPLOYMENT.md
+748
View File
@@ -0,0 +1,748 @@
# Deployment
How Volumen runs in production.
## Topology
```mermaid
flowchart TD
net["Internet"] -->|HTTPS| nginx
nginx["nginx :443, TLS termination and reverse proxy"] -->|"proxy_pass http://[::1]:9091"| app
app["volumen, one static binary, loopback only"] --> data
app --> etc
data["/var/lib/volumen holding posts/ (with media/ and .revisions/ inside) and users.toml, templates.toml, tokens.toml, webhooks.toml, secret.key"]
etc["/etc/volumen/config.toml"]
```
nginx terminates TLS on the public hostname and proxies `/api/volumen`,
`/admin` and `/media` to the backend on loopback; everything else on that
hostname is the public site, served separately. The engine owns one content
directory (posts, media and revisions inside it) and four files beside it:
`users.toml`, `templates.toml`, `tokens.toml` and `webhooks.toml`. There is
one process per content directory.
## Requirements
- **A supported platform**: the release builds cover Linux on `amd64`,
`arm64`, `loong64`, and `riscv64`, and FreeBSD on `amd64` and
`arm64`. The binary is static
(`CGO_ENABLED=0`), so no runtime libraries are needed.
- **Go 1.27.1** or newer only when building from source: the module declares
it.
- **nginx** and a TLS certificate (e.g. via certbot) for the public hostname.
- A dedicated system user, `volumen` by default, for the service. The
documentation creates it by hand; nothing in the binary creates users.
- **systemd** for the managed service (optional; an rc.d script, a supervisor,
or a plain foreground run all work).
- Nothing else: no interpreter, no virtual environment, no package manager.
The deployment itself is one command: start `volumen serve` and open
`/admin`, where the first-run wizard creates the administrator account. The
commented configuration template, `config.toml.example`, is committed at
the repository root, and the session secret is generated by the server.
## Build
```sh
just build # writes bin/volumen for the host platform
```
Releases are built by the tag pipeline for Linux (`amd64`, `arm64`, `loong64`,
`riscv64`) and FreeBSD (`amd64`, `arm64`), with `CGO_ENABLED=0` and
no build tags, so every artefact is a static binary. `-trimpath` keeps the
checkout's own path out of the binary and `-buildvcs=true` records the
revision it came from, so the same tree builds the same bytes from any
directory and `volumen version` still names the tag or the commit. The version
is recorded by the toolchain at build time; nothing injects it.
## Run
Installing the binary is a copy, and the installation is the wizard: start
`volumen serve`, open `/admin`, and the first-run screen creates the
administrator account, the interface language and the colour scheme. A
system deployment writes the config by copying `config.toml.example`; a
per-user one needs no config at all.
Any account can then add a second factor from Settings, Security: scan the
offered QR with an authenticator application and confirm one code. The
recovery codes shown at that moment open the account when the application
is lost; they work once each and are not shown again, so they belong in a
password manager the moment they appear.
### From a Gitea release
Every release on
[sourcedock.dev](https://sourcedock.dev/petrbalvin/volumen/releases) ships
platform binaries and a `checksums.txt` with their SHA-256 digests. Asset
names follow `volumen-<version>-<os>-<arch>`:
```sh
VERSION=1.0.0
BASE="https://sourcedock.dev/petrbalvin/volumen/releases/download/v${VERSION}"
curl -fLO "${BASE}/volumen-${VERSION}-linux-amd64"
curl -fLO "${BASE}/checksums.txt"
# Verify the download against the published digest.
sha256sum --ignore-missing -c checksums.txt
sudo install -m 0755 "volumen-${VERSION}-linux-amd64" /usr/local/bin/volumen
volumen version
```
Replace `linux-amd64` with `linux-arm64`, `linux-loong64`, `linux-riscv64`,
`freebsd-amd64`, or `freebsd-arm64`
as needed. The release binary carries its version, so `volumen version`
confirms what you installed.
### From source
```sh
git clone https://sourcedock.dev/petrbalvin/volumen.git
cd volumen
just install
sudo install -m 0755 bin/volumen /usr/local/bin/volumen
```
The binary reports the version the toolchain recorded at build time:
`volumen version` names the commit of the checkout it was built from, and a
release build names its tag. There is no version flag to pass and nothing to
inject.
### Bootstrap the deployment
There is no bootstrap command. A deployment is a directory for its state, a
configuration file copied from the template when the defaults do not fit, a
supervisor to keep the server running, and one visit to `/admin`, where the
first-run wizard creates the first account. The wizard stays open until that
account exists: on a machine reachable from the network, start the service and
claim the installation straight away.
### System install (root, systemd)
```sh
# 1. Service account and its group.
sudo useradd --system --user-group --home-dir /var/lib/volumen \
--shell /usr/sbin/nologin volumen
# 2. The config: copy the commented template from the repository root.
sudo mkdir -p /etc/volumen /var/lib/volumen/posts/media
sudo cp config.toml.example /etc/volumen/config.toml
# 3. Edit the copy for production behind a reverse proxy:
# host = "::1", env = "production", trust_proxy = true,
# cookie_secure = true,
# trusted_proxies = ["::1", "127.0.0.1"]
# Leave [admin].session_key empty: the server generates its secret into
# /var/lib/volumen/secret.key on first start and keeps it there.
# 4. Own the data directory so the service can write posts, media, accounts
# and its secret.
sudo chown -R volumen:volumen /var/lib/volumen
# 5. Install the unit (the Service unit section below), then:
sudo systemctl daemon-reload
sudo systemctl enable --now volumen.service
# 6. Open http://localhost:9091/admin/ and complete the wizard.
```
The two proxy settings carry weight: `trust_proxy = true` takes the client
address from the proxy's `X-Forwarded-For`, and `cookie_secure = true` keeps
the session cookie on HTTPS. The config is deployment state and stays mode
`0600`.
### Per-user install (no root)
```sh
volumen serve
```
With no config file the server runs on the per-user paths:
`~/.local/share/volumen/posts` and `~/.local/share/volumen/users.toml`
(honouring `XDG_DATA_HOME`), and it reads `~/.config/volumen/config.toml`
when that file exists. No systemd unit is installed in this mode; run the
server in the foreground or through a supervisor of your choice.
### Configuration locations
| Path | Purpose |
|------|---------|
| `/etc/volumen/config.toml` | configuration: a copy of the commented `config.toml.example`, edited to fit |
| `/var/lib/volumen/posts` | Markdown posts |
| `/var/lib/volumen/posts/media` | uploaded images (WebP / AVIF / SVG) |
| `/var/lib/volumen/posts/.revisions` | archived post versions and delete tombstones |
| `/var/lib/volumen/users.toml` | admin users (created by the first-run wizard or the Settings page) |
| `/var/lib/volumen/secret.key` | the session secret the server generated on first start |
| `/var/lib/volumen/templates.toml` | post templates (created from the admin Settings page) |
| `/var/lib/volumen/tokens.toml` | API access tokens (created from the admin Settings page) |
| `/var/lib/volumen/webhooks.toml` | admin-managed webhook endpoints (created from the admin Settings page) |
| `/var/lib/volumen/audit.log` | audit trail (only when `audit_log` is configured) |
| `/etc/systemd/system/volumen.service` | systemd unit (written by the operator, see the Service unit section) |
| `~/.config/volumen/config.toml` | per-user config, read when it exists |
| `~/.local/share/volumen/` | per-user data dir (the default when no config file exists) |
The users, templates, tokens and secret files carry password hashes, token
digests and the signing key, so never make them world-readable: every write
the server performs re-applies mode `600`, media and post writes are atomic.
Every configuration key is documented in
[CONFIGURATION.md](CONFIGURATION.md).
### Scheduled publishing
Posts may carry a `publish_at` date in their frontmatter; they stay hidden
until that date arrives. Two mechanisms can flip them, and both do the same
thing: drop `publish_at` and set `date` when the post had none.
**From cron** (or a systemd timer):
```text
*/5 * * * * /usr/local/bin/volumen publish-due --config /etc/volumen/config.toml
```
Use the absolute path to the binary, since cron's `PATH` is minimal.
`--dry-run` lists due posts without touching files; `--json` emits
machine-readable output. A real run delivers the `post.published` event to the
configured `[[webhooks]]` for every post it publishes and waits for those
deliveries to finish, so a front end that rebuilds from a webhook hears about
a publish that came from cron. A post whose file could not be saved is
reported on stderr and the command exits `1`, so a timer unit or a monitoring
check can see the failure rather than assume success.
**From the server itself**, via the `[scheduler]` configuration section:
```toml
[scheduler]
enabled = true
interval = 300 # seconds
```
The in-process loop starts with `volumen serve` and needs no external timer.
It sweeps once at start-up, so a post whose date passed while the service was
down is published as soon as it comes back, and then once every `interval`
seconds. An interval below one second never reaches the loop: the
configuration validation refuses it at start-up. The loop delivers the same
`post.published` webhook as the
admin does, and a save failure is logged and skipped, so one bad file cannot
stop the sweep or the loop.
The systemd timer equivalent for the CLI path:
```ini
# /etc/systemd/system/volumen-publish.service
[Unit]
Description=Publish due volumen posts
[Service]
Type=oneshot
User=volumen
ExecStart=/usr/local/bin/volumen publish-due --config /etc/volumen/config.toml
```
```ini
# /etc/systemd/system/volumen-publish.timer
[Unit]
Description=Publish due volumen posts every five minutes
[Timer]
OnCalendar=*:0/5
[Install]
WantedBy=timers.target
```
```sh
sudo systemctl daemon-reload
sudo systemctl enable --now volumen-publish.timer
```
Adjust the `ExecStart` path to match the one in `volumen.service`.
## Service unit
`/etc/systemd/system/volumen.service`, written by the operator (paths and
binary location fitted to the deployment):
```ini
[Unit]
Description=Volumen, a lightweight publishing platform for scientists
After=network.target
[Service]
Type=simple
User=volumen
Group=volumen
WorkingDirectory="/etc/volumen"
ExecStart="/usr/local/bin/volumen" serve --config "/etc/volumen/config.toml" --content "/var/lib/volumen/posts"
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths="/var/lib/volumen"
PrivateTmp=true
[Install]
WantedBy=multi-user.target
```
Quote every path so a layout with spaces in it stays one argument, and double
any `%`, because unit files expand `%X` specifiers.
- `User` / `Group` name the service account (default `volumen`), and
`WorkingDirectory` is the directory holding the config file.
- `ExecStart` names the installed binary.
- `ReadWritePaths` is the **parent of the data directory**, so a layout with
posts at `/srv/volumen/posts` yields `ReadWritePaths=/srv/volumen`: the
service also writes the users file, its templates, tokens, webhooks and
`secret.key` there. With `ProtectSystem=strict` the rest of the filesystem
is read-only to the service.
- `NoNewPrivileges`, `ProtectHome` (`/home`, `/root`, `/run/user`
inaccessible), `PrivateTmp`, and a two-second restart delay on failure.
The server bounds every connection: a 10 second read-header timeout, a 60
second read timeout, a 120 second write timeout, and a 120 second idle
timeout, so a client that opens a socket and dribbles a request cannot hold it
indefinitely. On `SIGINT` or `SIGTERM` (what `systemctl stop` sends) it stops
accepting new connections and drains the requests already in flight, giving
them at most 15 seconds.
### FreeBSD (rc.d)
The `freebsd/amd64` and `freebsd/arm64` release binaries
run natively; nothing is compiled at install time.
```sh
# Install the binary from the release, as above, to /usr/local/bin/volumen.
pw useradd volumen -d /var/db/volumen -s /usr/sbin/nologin -c "Volumen publishing platform"
mkdir -p /usr/local/etc/volumen /var/db/volumen/posts/media
cp config.toml.example /usr/local/etc/volumen/config.toml
chown -R volumen /var/db/volumen
```
Edit the copied config for a production deployment behind a proxy: set the
three paths, `host = "::1"`, `env = "production"`, `trust_proxy = true`,
`cookie_secure = true` and `trusted_proxies = ["::1", "127.0.0.1"]`. Leave
`[admin].session_key` empty; the server writes `secret.key` into
`/var/db/volumen`, which is owned by the `volumen` user.
Conventional FreeBSD paths: config in `/usr/local/etc/volumen/`, data in
`/var/db/volumen/`. Drop an `rc.d` script in
`/usr/local/etc/rc.d/volumen`:
```sh
#!/bin/sh
#
# PROVIDE: volumen
# REQUIRE: NETWORKING
# KEYWORD: shutdown
. /etc/rc.subr
name="volumen"
rcvar="volumen_enable"
load_rc_config $name
: ${volumen_enable:="NO"}
: ${volumen_user:="volumen"}
: ${volumen_config:="/usr/local/etc/volumen/config.toml"}
: ${volumen_content:="/var/db/volumen/posts"}
pidfile="/var/run/${name}.pid"
command="/usr/sbin/daemon"
command_args="-f -r -P ${pidfile} -u ${volumen_user} \
/usr/local/bin/volumen serve --config ${volumen_config} --content ${volumen_content}"
run_rc_command "$1"
```
Enable and start:
```sh
chmod +x /usr/local/etc/rc.d/volumen
sysrc volumen_enable=YES
service volumen start
```
## Production configuration
Volumen only owns `/api/volumen`, `/admin`, and `/media`. Serve your public
site from the same hostname and proxy those prefixes to the backend, so the
admin runs same-origin (session cookies and CSRF work without extra
configuration). A production deployment binds the service to `::1`, so proxy
to `[::1]:9091`:
```nginx
server {
listen 443 ssl http2;
server_name lab.example.com;
ssl_certificate /etc/letsencrypt/live/lab.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/lab.example.com/privkey.pem;
# Public site: any front end, static files or a built one.
root /var/www/site;
location / {
try_files $uri $uri/ /index.html;
}
# volumen API, admin and media.
location ~ ^/(api/volumen|admin|media)(/|$) {
proxy_pass http://[::1]:9091;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
server {
listen 80;
server_name lab.example.com;
return 301 https://$host$request_uri;
}
```
Caddy is the recommended front, and the same shape costs a handful of
lines: certificates and the HTTP redirect are its own work.
```caddyfile
lab.example.com {
root * /var/www/site
@backend path /api/volumen/* /admin /admin/* /media /media/*
handle @backend {
reverse_proxy [::1]:9091
}
handle {
try_files {path} {path}/ /index.html
}
}
```
Caddy appends the connecting address to `X-Forwarded-For` the way
`$proxy_add_x_forwarded_for` does, so the guarantee described below holds
with either front.
With `[server].trust_proxy = true` (a production deployment turns it on), the
client IP used by the API rate limiter and the login limiter is the **last**
entry of `X-Forwarded-For`, and only when the connection itself comes from an
address listed in `[server].trusted_proxies`. Session cookies then always
carry `Secure`.
The last entry is the one the peer that wrote it saw as its client, so the
directives above are safe as written: `$proxy_add_x_forwarded_for` appends
`$remote_addr` after anything the client sent, which means a client cannot
displace its own address by sending a header of its own. The guarantee the
proxy must provide is the other half: only nginx may reach the backend. Bind
it to `::1`, and never expose port 9091,
because anything that can open a connection to the backend directly can set
`X-Forwarded-For` itself and choose the address it is rate-limited under (see
the security notes below).
`[server].trusted_proxies` turns that guarantee into a check rather than a
promise: with `trusted_proxies = ["::1", "127.0.0.1"]`, the forwarded address
is believed only when the connection itself comes from the loopback proxy, and
a request that arrives from anywhere else is measured by its real address.
A loopback proxy behind `nginx` uses exactly that list. An empty list never reads
the header at all: behind a proxy every client is then measured under the
proxy's own connection address and shares one rate-limit budget, so list the
proxy to measure each client by its forwarded address.
### Hardening the login
The engine rate-limits `/admin/login` itself (10 attempts per IP per 60 s;
see [the session and rate-limit rules](ARCHITECTURE.md#state-and-lifetime)),
but nginx is the durable line of defence. Add brute-force throttling (and,
optionally, an IP allowlist) at the proxy.
**1. A rate-limit zone** in the `http { }` context (e.g.
`/etc/nginx/conf.d/volumen.conf`):
```nginx
limit_req_zone $binary_remote_addr zone=volumen_login:10m rate=5r/m;
```
**2. A reusable proxy snippet** at `/etc/nginx/snippets/volumen-proxy.conf`:
```nginx
proxy_pass http://[::1]:9091;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
```
**3. Split the single proxy location** in the `server { }` block into three, namely
public API/media, the throttled login, and the rest of the admin:
```nginx
# Public API and uploaded media, open to everyone.
location ~ ^/(api/volumen|media)(/|$) {
include snippets/volumen-proxy.conf;
}
# Login: throttle brute-force attempts on top of the app-level limit.
location = /admin/login {
# Optional IP allowlist (uncomment to restrict):
# allow 203.0.113.0/24;
# deny all;
limit_req zone=volumen_login burst=10 nodelay;
include snippets/volumen-proxy.conf;
}
# The rest of the admin.
location ^~ /admin {
include snippets/volumen-proxy.conf;
}
```
nginx matches `= /admin/login` (exact) first, then `^~ /admin` (which stops
regex matching), then the `~` regex for API/media. Apply with
`sudo nginx -t && sudo systemctl reload nginx`.
### Security notes
- Bind to `::1` (or `127.0.0.1`); only nginx is public. TLS is terminated by
nginx. The production config sets `host = "::1"`.
- Session cookies are `HttpOnly` and `SameSite=Strict`, and gain the `Secure`
attribute when `[server].cookie_secure = true` or when `trust_proxy = true`.
The session secret signs them and survives restarts: by default the server
generates it into `secret.key` beside the users file, and `[admin].session_key`
(at least 64 bytes, mandatory in production) overrides that file.
- `[server].trusted_proxies` names the addresses whose `X-Forwarded-For` is
believed. A loopback-nginx deployment lists `["::1", "127.0.0.1"]`, or narrows
it to the proxy's address, so a client that reaches the backend directly cannot
choose the address it is rate-limited under. An empty list ignores the
forwarded header, which behind a proxy folds every client into one
rate-limit bucket.
- Keep `config.toml`, `users.toml`, `tokens.toml` and `secret.key` readable only
by the `volumen` user (mode `600`). Every write the server makes enforces this.
- To change a forgotten administrator password, reset it from the admin Settings
page as another admin, or rotate the account in `users.toml`. Never put a
password on the command line.
- The controls themselves, and their limits, are described in
[the architecture](ARCHITECTURE.md#state-and-lifetime): how a body is
sanitised before it is served, how a password is stored, and what the session
cookie does and does not guarantee.
## Upgrade
### From the admin panel
The Version panel in Settings shows the running release and, when the release
check has found a newer one, an "Update now" action. The update endpoint is
admin-only and CSRF-protected. On confirmation the server:
1. queries the latest release from the Gitea releases API
(`/api/v1/repos/petrbalvin/volumen/releases/latest`),
2. downloads the matching asset from
`https://sourcedock.dev/petrbalvin/volumen/releases/download/v<version>/volumen-<version>-<os>-<arch>`,
3. verifies the SHA-256 digest against the release's `checksums.txt`,
4. stages the new binary in the executable's directory and renames it over the
running one,
5. re-executes itself with the original command-line arguments, preserving the
process ID so systemd never observes a restart.
The response page polls `/healthz` until the server is back (up to three
minutes). Nothing is installed without an admin clicking through.
The self-update writes the running binary in place, so the service user needs
write access to that directory. With the standard layout
(`/usr/local/bin/volumen` owned by root, service running as `volumen`) the
rename fails; either grant the service user write access to the install
directory or use the manual path below. A checksum or download error in the
panel means the release asset could not be fetched for the running platform;
the state on disk is untouched.
### From the shell
```sh
sudo systemctl stop volumen
sudo install -m 0755 volumen-1.0.0-linux-amd64 /usr/local/bin/volumen
volumen version
sudo systemctl start volumen
volumen status # confirm: config, posts, users
```
The persistent state (`config.toml`, `users.toml`, posts, revisions) is
untouched by either path.
### Monitoring for updates
```sh
volumen check-update # human-readable
volumen check-update --json # machine-readable
```
Exit codes: `0` when the running version is the latest release, `1` when a
newer release exists, and `1` as well when the release API cannot be reached
(the cause is on stderr, so a monitoring check distinguishes the two by its
output rather than by the code); `2` is a usage error and nothing else.
## Rollback
There is no rehearsed rollback procedure for a running installation, and this
document does not pretend otherwise.
- **A bad upgrade** is undone by putting the previous binary back and
restarting: the release assets are versioned, so an older one is still on the
releases page, and the data is untouched by an upgrade. A restore from a
backup archive is the fallback when data changed as well.
- **A bad content change** does not need a rollback: every save archives the
previous version under `posts/.revisions/<slug>/`, and a delete is a move into
that archive, so the admin's History view restores either one.
- **A bad configuration change** is undone by editing the file and restarting;
a configuration that fails validation stops the process before it binds its
port, so a broken edit cannot half-start the service.
### Backups
All state is files, so any filesystem backup works. Volumen also ships its
own archive commands:
```sh
volumen export --out /backup/volumen-$(date +%F).tar.gz
volumen import /backup/volumen-2026-08-02.tar.gz
```
`export` packs the content directory (posts, media, `.revisions`) under
`posts/`, plus `users.toml`, `templates.toml` and `tokens.toml`, into a
`tar.gz` written at mode `600`; `--out` is the flag, and the default output
path is `volumen-backup.tar.gz` in the working directory. `config.toml` is not
in the archive: the import has no use for it and the session key it carries is
a credential that should not travel in a file copied around. `import` restores
the posts, users, templates and tokens into the locations from the config.
A restore is confined by construction: the archive's entries are matched
against the fixed `posts/`, `users.toml`, `templates.toml` and `tokens.toml`
names, a `posts/` path must be a post, a revision or a media file with an
allowed image extension, and every write goes through an `os.Root` opened on
the content directory, which refuses an escape through `..` or through a
symlink. The decompressed size is bounded at 512 MiB, so a compression bomb
cannot exhaust memory. `.toml` files are written with mode `600`, everything
else with `644`.
The same export and restore is available from the admin Settings page, where
both directions require the **admin** role, because the archive carries the
users file with its password hashes and the tokens file with its digests. The
admin path calls the same two functions as the CLI, so the archive and the
containment rules are one implementation, not two.
## Monitoring
After the first-run wizard, confirm the installation state:
```sh
volumen status # config, content dir, post and user counts
volumen status --json # machine-readable; exit code 1 on any issue
volumen doctor # config validation, post rendering, weak hashes
volumen doctor --json # machine-readable; exit code 1 when a check fails
volumen validate # content check: slugs, titles, aliases, rendering
sudo systemctl status volumen
journalctl -u volumen -f
```
`volumen status` reads the config and reports the post count (with a draft
count) and the number of users; with no accounts yet it names the wizard as
the next step. `volumen doctor` validates the configuration,
renders every post, and warns when a stored password hash uses scrypt
parameters below the current policy floor. The exit code follows the levels:
a missing or broken config, an unreadable content directory and an unreadable
users file fail the command with exit code 1, while an installation still
waiting for its first account, posts that fail to render
and weak stored hashes are warnings that leave the exit code at 0.
`volumen validate` reports
duplicate slugs, invalid slugs, missing titles, aliased collisions, and posts
that fail to render.
Those commands check a deployment. The build of the binary is checked by
`just gates`, which runs `build`, `fmt-check`, `vet`, `test` and `race` in one
pass; the release pipeline runs the same set without the race detector (see
[Pipeline](#pipeline)).
Logs go to stderr and therefore to journald under systemd. For structured
output, set the log format to JSON:
```toml
[server]
log_format = "json"
```
Each line is then one JSON object from `log/slog` (time, level, message, and
the structured attributes of the event), suitable for `journalctl -o json`,
Loki, or similar. The default `"text"` format is human-readable. Every request
is logged once when it finishes, with its method, path, status and duration,
under a 16-character request id; the same id is answered in the `X-Request-Id`
header and attached to every line the handlers write while serving that
request, so one id finds a request and everything it did in the journal. The
server's own protocol errors are written through the same logger rather than
straight to stderr.
Set `audit_log` in the configuration to keep a separate, append-only
JSON-lines record of administrative actions. Logins and logouts are not
recorded; post creation, edits, deletions and bulk actions are, together with
media deletions, user and token management, and backup imports. Each line is
one JSON object carrying the timestamp `ts` (RFC 3339, UTC), the acting `user`
and the `action` as its message, plus the `resource`, the client `ip` and a
`detail` object wherever the action knows them.
## Pipeline
Releases are built by `.gitea/workflows/release.yml`, which runs when a `v*`
tag is pushed. The branch flow is the one in
[CONTRIBUTING.md](../CONTRIBUTING.md): work lands on `development`,
`development` is merged into `main`, and the tag is cut on `main`. The build
happens at the tag, and the toolchain records the tag into the binary's build
information, so the version in the binary is right because of where the build
ran; nothing is injected, and each job derives the version from the tag itself
rather than receiving it from another job.
```mermaid
flowchart TD
subgraph gates["gates job, 10 minute timeout"]
direction TB
g1["validate the tag against the semver pattern"] --> g2["go build ./..."] --> g3["gofmt -l . prints nothing"] --> g4["go vet ./..."] --> g5["go fix -diff ./..."] --> g6["go test with a coverage profile"] --> g7["the coverage floor of 80 per cent"]
end
subgraph builds["build job, 25 minute timeout, six targets in one matrix"]
direction TB
b1["linux on amd64, arm64, loong64, riscv64"] --> b2["freebsd on amd64, arm64"]
b2 --> b3["CGO_ENABLED=0 build with -trimpath and -buildvcs into bin/volumen-VERSION-OS-ARCH"]
b3 --> b4["upload the artefact"] --> b5["linux/amd64 smoke test, the binary reports the tag and no +dirty"]
end
subgraph rel["release job, 15 minute timeout"]
direction TB
r1["download the artefacts"] --> r2["write checksums.txt over them"] --> r3["take the CHANGELOG section for the tag"] --> r4["create the release over the Gitea API"] --> r5["upload the six binaries and checksums.txt"]
end
tag["a v* tag is pushed"] --> gates
gates --> builds
builds --> rel
```
The gates job runs the same set as `just gates` minus the race detector:
build, format check, `go vet` together with `go fix -diff`, the suite, and the
coverage floor of 80 per cent. Race is deliberately absent: the runner is
shared with the forge, and `just gates` races the tree on the machine where
the tag is cut. A push or a pull request to `development` runs the same steps
without race through `.gitea/workflows/test.yml`, and
`.gitea/workflows/race.yml` runs the suite under the race detector when it is
dispatched by hand.
The release is the only publisher: nothing is deployed out of the repository,
and a production installation takes its binary from a release, through the
admin self-update (see [Upgrade](#upgrade)) or a manual install followed by a
service restart.
Each matrix entry builds one static binary named
`volumen-<version>-<os>-<arch>` for the version without its leading `v`, with
`-trimpath` and `-buildvcs=true`, uploads it, and, on linux/amd64 only, runs it
and requires the output of `volumen version` to contain the tag and not
`+dirty`. The release job then
writes `checksums.txt`, one `sha256sum` line per asset, takes the release
notes from the `CHANGELOG.md` section for the tag, creates the release over
the Gitea API, and uploads the six binaries with the checksums file. That
file is part of the update contract: the admin self-update refuses a download
whose SHA-256 does not match it.
The release carries no licence files of its own, because the repository holds
them: `LICENSE` for Volumen, and
[NOTICE.md](../NOTICE.md) for every module the binary
is compiled from. Gitea attaches the tag's own source archive to the release
beside the binaries, and that archive contains both.
docs/DEVELOPMENT.md
+161
View File
@@ -0,0 +1,161 @@
# Development
How to work on **Volumen**.
## Prerequisites
- Go 1.27.1, the newest stable release; the `go` directive in `go.mod`
declares exactly that version, and nothing older builds the module.
- [just](https://github.com/casey/just) for the recipes.
- gcc, but only for the race detector: `just race` needs cgo. The build
itself is pure Go with `CGO_ENABLED=0`.
- Perl for the scripted recipe lines and `scripts/notices.pl`; the base
interpreter with builtins only is enough.
## Setup
```sh
git clone https://sourcedock.dev/petrbalvin/volumen.git
cd volumen
just build
```
`just run` starts the server against `./config.toml` and `./posts` on the
configured host and port (default `[::]:9091`); `just dev` binds it to
`127.0.0.1:9091`. `config.toml` and `users.toml` are git-ignored, because the
config holds secrets: on a fresh clone the server runs on the built-in
defaults and the overridden content directory. Both recipes pass
`-buildvcs=true`, because plain `go run` does not stamp the build, so the
reported version would be `(devel)`. Go has no built-in reload; restart the
process after code changes. The whole binary rebuilds in seconds, which
avoids waiting for a reload cycle.
## Recipes
Every recipe in the project's file, and what it does. Taken from the file
itself, so the names and the list match it exactly, in the order the file
declares them.
| Recipe | What it does |
|---|---|
| `just` | the `default` recipe, `just --list`: every recipe with its one-line summary |
| `just build` | compiles `cmd/volumen` into `bin/volumen` with `CGO_ENABLED=0`, `-trimpath` and `-buildvcs=true`, stripped, zero errors and zero warnings |
| `just test` | the full suite as CI runs it, with the 80 percent coverage floor |
| `just race` | the same suite under the race detector; the expensive one |
| `just unit ./internal/store 'TestName'` | a scoped run for iterating, cached, no race, no coverage |
| `just fuzz <target> <package>` | a time-boxed fuzz of one target in one package, never a gate |
| `just bench` | benchmarks with `-benchmem` and five counts, on an idle machine only |
| `just fmt` | gofmt over the tree, in place |
| `just fmt-check` | zero diff; prints nothing when everything is formatted |
| `just vet` | `go vet` and `go fix -diff` |
| `just gates` | the definition of done in one command: `build`, `fmt-check`, `vet`, `test`, `race`, in that order |
| `just clean` | removes `bin/` and `coverage.out` |
| `just install` | builds, then copies the binary into `~/.local/bin` |
| `just uninstall` | removes the installed binary |
| `just run` | runs the server against `./config.toml` and `./posts`, with the build stamped |
| `just dev` | the same, bound to `127.0.0.1:9091` |
`just fmt` and `just fmt-check` are the formatting authority, `just vet` the
static one, and the three of them plus `build`, `test` and `race` are the
whole gate. Style rules the tools cannot see (error wrapping, `log/slog`
only, no panics outside `main`, British English names and comments) are
written in [CONTRIBUTING.md](../CONTRIBUTING.md).
## Running a single test
```sh
go test -run TestName ./internal/store
```
Narrow selections do not apply the coverage floor (that lives in the `test`
recipe), so use them freely while iterating. Or stay on the recipe:
`just unit ./internal/store 'TestName'`. Add `-v` for the sub-test names,
`-race` when the change touches concurrency, and `-count=1` when a cached
result looks stale; the gate itself runs with `-count=1`, so a cached pass
never stands in for a fresh one.
The suite mirrors the package layout (`*_test.go` next to the code), shares
no global fixtures, and uses `t.TempDir()` and `net/http/httptest` rather
than a real network:
| Package | Covers |
|---|---|
| `cmd/volumen` | CLI dispatch and every subcommand against temp directories. |
| `internal/admin` | Login, CSRF, post CRUD, settings, media, roles. |
| `internal/app` | Server assembly, the route table, `/healthz`, `robots.txt`, media serving, rate-limit headers. |
| `internal/audit` | The JSON-lines audit log, and a disabled or unwritable one. |
| `internal/backup` | Archive round-trips, the restore allowlist and the decompression budget. |
| `internal/biblio` | The `refs` frontmatter into a numbered, linked reference list. |
| `internal/config` | Loading, merging, defaults, overrides, validation, and the example file matching the embedded template. |
| `internal/diff` | The line-based LCS diff behind the revision comparison view. |
| `internal/fediverse` | The `@user@host` handle rule. |
| `internal/identifiers` | The scholarly identifier rules: DOI syntax and normalisation, ORCID shape and ISO 7064 check digit. |
| `internal/feeds` | RSS, Atom, JSON Feed and sitemap output. |
| `internal/frontmatter` | TOML frontmatter parsing, key order and round-trips. |
| `internal/httpapi` | The public API over `httptest`: ETags, CORS, pagination, token writes, and the committed contract fixtures. |
| `internal/i18n` | The admin interface catalogue: English keys, Czech translations, plural rules. |
| `internal/imagefile` | The accepted extensions, the WebP, AVIF and SVG signatures, the detected type and the header dimensions. |
| `internal/markdown` | Rendering, sanitisation, the mathematics and diagram passes, the table of contents, and the golden corpus in `testdata/`. |
| `internal/password` | scrypt hashing and verification. |
| `internal/payloads` | Payload builders, filtering and validation. |
| `internal/post` | The Post model: parsing, metadata, rendering, cloning. |
| `internal/preview` | The signed share links for unpublished posts. |
| `internal/ratelimit` | The sliding-window limiter. |
| `internal/scheduler` | Due-post selection, the publish sweep and the interval loop. |
| `internal/session` | HMAC-signed cookies and middleware. |
| `internal/store` | Loading, saving, deleting posts, revisions, tombstones, media. |
| `internal/templates` | The `templates.toml` post templates. |
| `internal/tokens` | Minting, scopes, authentication and revocation. |
| `internal/tomlfile` | Atomic `0600` TOML writes and the shape helpers. |
| `internal/updater` | Release checks, version comparison and the verified self-update. |
| `internal/users` | `users.toml`, roles and last-admin safeguards. |
| `internal/version` | The release identity from `debug.ReadBuildInfo`. |
| `internal/web` | Middleware: gzip, the cross-origin gate, security headers, the CSP nonce, the request logger and the client IP. |
| `internal/webhooks` | Signed deliveries, retries, payload shapes. |
## Coverage
```sh
just test
go tool cover -func=coverage.out
```
The `total:` line is the number that matters, and it stays at 80 percent or
more; the `test` recipe fails below the floor.
## Benchmarks
```sh
just bench
```
The recipe is `go test -run '^$' -bench=. -benchmem -count=5 ./...` over the
whole module. Two benchmarks live in the tree, `BenchmarkRender` in
`internal/markdown` and `BenchmarkServer` in `internal/app`; the binding
measurement method is in [BENCHMARKING.md](BENCHMARKING.md). Benchmark on an
idle machine, and compare only runs made in one process against each other.
## Debugging the build
```sh
go build -gcflags='-m' ./... # inlining decisions
go build -gcflags='-S' ./... # what the compiler generated
```
## Continuous integration
Workflows live in `.gitea/workflows/` and run on the project's own runners.
They are written by hand rather than through `just`, but they enforce the same
set of gates, so a green `just gates` locally is the fastest way to a green
pipeline.
## Releases
Releases are cut by merging `development` into `main` and tagging `vX.Y.Z`. The
tag drives the release workflow, which builds the assets and publishes the
notes it extracted from `CHANGELOG.md`.
Dependency changes touch one more file: `NOTICE.md` in the root
reproduces the licence of every module the binary is compiled from, and
`perl scripts/notices.pl` writes it again after a module is added, removed or
upgraded.