scripts/docs/DEPLOYMENT.md

# Deployment

The collection has two senses of deployment: how the scripts themselves are published,
and what they deploy when they run on a host. Both are below.

## Topology: publishing

The scripts are served from `https://petrbalvin.org/scripts/`, which is a directory on a
host reached over SSH. The Gitea instance at `sourcedock.dev` holds the repository and
runs the pipeline; there is no package, no tag and no artefact store, because a script
that is a single file is published by copying it.

```mermaid
flowchart LR
    Dev[development branch] -->|merge| Main[main]
    Main --> Runner[Gitea runner, alpine]
    Runner -->|rsync over ssh| Host[petrbalvin.org]
    Host -->|https| Client[client]
    Runner -->|SHA256SUMS| Host
```

## Requirements

- The runner image is `alpine`, which carries `ssh`, `sshpass`, `rsync` and no Perl: the
  pipeline installs Perl in its first step.
- Five repository secrets: `DEPLOY_HOST`, `DEPLOY_USER`, `DEPLOY_PATH`,
  `DEPLOY_PASSWORD`, and `DEPLOY_KNOWN_HOSTS`.
- `DEPLOY_KNOWN_HOSTS` holds the host key, from
  `ssh-keyscan -t ed25519 <host>`. The pipeline writes it into `~/.ssh/known_hosts` and
  connects with `StrictHostKeyChecking=yes`; without the secret the deploy stops before
  it starts, deliberately, because the alternative is trusting whatever answers on the
  first connection.

## Build

There is nothing to build. The pipeline writes `SHA256SUMS`, one line per script, hashed
with `sha256sum` in sorted filename order so the file is stable between runs, and then
sends the scripts and that file.

## Run

The pipeline runs itself: merging into `main` publishes, and a manual dispatch republishes
the current tree.

```mermaid
sequenceDiagram
    participant Dev
    participant CI as Gitea runner
    participant Host as petrbalvin.org
    Dev->>CI: push to main
    CI->>CI: write SHA256SUMS over every script
    CI->>CI: pin the host key from DEPLOY_KNOWN_HOSTS
    CI->>Host: rsync -avz --delete, the scripts and SHA256SUMS
    Host-->>CI: exit status of rsync
    CI->>CI: stop the run unless it was zero
```

`--delete` is deliberate: the host directory mirrors the repository, so a script removed
here disappears there. Nothing else lives in that directory.

## Upgrade and rollback

A publish is atomic per file as rsync writes it, and the previous copy is overwritten in
place. Rollback is a revert commit on `main`: the pipeline publishes the reverted tree on
the next push, and the host is back to the earlier scripts within the minute the run
takes. There is no rehearsal of that procedure on a staging host; it is the same
pipeline with the same single target.

## Monitoring

Nothing polls the host. Two things are worth watching by hand:

- The pipeline's own run history in Gitea, which fails loudly on a non-zero `rsync` or a
  missing secret.
- `https://petrbalvin.org/scripts/SHA256SUMS`, which a client can compare against what it
  downloaded. A file that is on the host but missing from `SHA256SUMS` means a publish was
  interrupted.

## What the scripts deploy

Two of the six change a host's role, and a third prepares a desktop. Their output is the
deployment an operator cares about, so the pipeline that publishes them never runs them.

| Script | What it leaves behind | Where it is documented |
|---|---|---|
| `server-setup.pl` | Packages, firewalld with the SSH rule already allowed, SELinux enforcing, Podman, unattended updates | [docs/CLI.md](CLI.md) |
| `sglang-deploy.pl` | A systemd unit running the engine in a Podman container, nginx with TLS in front, an API key file, the firewall rule and the SELinux boolean | [docs/ARCHITECTURE.md](ARCHITECTURE.md) |
| `workstation-setup.pl` | A Fedora desktop with the toolchains and applications installed | [docs/CLI.md](CLI.md) |

The SGLang deployment is the only service in the collection, and the host needs no ROCm
installation for it: the engine runs from the project's ROCm image and reaches the GPU
through `/dev/kfd` and `/dev/dri`, which the amdgpu kernel driver provides. The image tag
follows the card and the host's ROCm, which is why the script resolves it rather than
carrying a constant.

## Production configuration

Nothing in this repository holds a secret. The values that differ per host come from
where the script reads them:

- The published copies are unversioned: whatever is on `main` is what is served.
- `sglang-deploy.pl` keeps the engine's API key in `/etc/sysconfig/sglang`, mode 0600,
  written by the script and never committed. A Hugging Face token for a gated model goes
  into that same file as `HF_TOKEN`, which the unit forwards to the container.
- The deploy secrets live in the Gitea repository settings, under Actions, Secrets.