149 lines
6.1 KiB
Markdown
149 lines
6.1 KiB
Markdown
# Development
|
|
|
|
How to work on the scripts collection.
|
|
|
|
## Prerequisites
|
|
|
|
- Perl 5.38 or newer, which is what the oldest supported system ships. Measured on the
|
|
three test platforms: Fedora 44 carries 5.42.3, CentOS Stream 10 carries 5.40.2, and
|
|
openEuler 24.03 LTS carries 5.38.0.
|
|
- Podman, for the container rigs that verify a script against a real system.
|
|
- Nothing else. There is no build step, no dependency to install and no module to fetch:
|
|
the scripts use the interpreter's builtins, and the checks use nothing beyond them
|
|
either.
|
|
|
|
## Setup
|
|
|
|
```sh
|
|
git clone https://sourcedock.dev/petrbalvin/scripts.git
|
|
cd scripts
|
|
perl -c network-diag.pl
|
|
```
|
|
|
|
## Commands
|
|
|
|
There is no recipe file: the gates are the commands below, and the test pipeline runs
|
|
the same set.
|
|
|
|
| Command | What it does |
|
|
|---|---|
|
|
| `perl -c <script>` | Compiles one script. All six in a loop is what the pipeline's first gate does |
|
|
| `perl tests/<script>.pl` | One script's checks, each printing `ok` or `FAIL` and exiting non-zero on a failure |
|
|
| `perl tests/container/rig.pl` | The container rig, which must be run through Podman as root: see below |
|
|
|
|
The whole local gate, one line per script:
|
|
|
|
```sh
|
|
for f in *.pl; do perl -c "$f" || exit 1; done
|
|
for t in tests/*.pl; do perl "$t" || exit 1; done
|
|
```
|
|
|
|
Two more gates have no command of their own because the pipeline carries them: every
|
|
script opens with the shebang `#!/usr/bin/env perl` and the two-line licence header,
|
|
and no file in the repository uses a dash as punctuation. Both are checked in
|
|
`.gitea/workflows/test.yml`.
|
|
|
|
## The checks under tests/
|
|
|
|
One file per script, named after it, run as `perl tests/<script>.pl`. They cover the
|
|
pure functions, which is where the arithmetic and the parsing live: RPM's version
|
|
comparison (checked against `rpm`'s own implementation, which is the definition of the
|
|
ordering), the two JSON decoders and the encoder, IPv4 and IPv6 parsing with the
|
|
`sockaddr` packing and the address classification, the ping transcript parser, the
|
|
grading, the dnf-automatic renderer, the checksum reader, the writers, and the argument
|
|
parsing of every script.
|
|
|
|
They need no root, no network and no Podman, except where a case is skipped for that
|
|
reason and says so: `atomic_write` settles the owner of the file it writes, so it can
|
|
only be exercised as root.
|
|
|
|
## The container rig
|
|
|
|
`tests/container/rig.pl` runs the real `sglang-deploy.pl` as root inside a container
|
|
against a stub `PATH`: every command the script drives (`dnf`, `rpm`, `podman`,
|
|
`systemctl`, `curl`, `openssl`, `caddy`, `tar`, `useradd`, `semodule`, `firewall-cmd`,
|
|
`getsebool`, `lspci`) is a stub that answers from a fixture, so a run is deterministic
|
|
and needs no network and no GPU. It covers the deploy end to end: the resolved image
|
|
tag, the unit file, the caddy drop-in and the main Caddyfile's import, the release
|
|
binary install for the hosts without a caddy package, the EPEL bootstrap on CentOS
|
|
Stream, the legacy nginx clean-up, the TLS certificate and its modes, the API key file,
|
|
the SELinux decisions, the firewall rule, the idempotent second run, the dry run, the
|
|
uninstall, the Radeon and MI300 paths, the offline and unpublished-tag failures, the
|
|
argument validation and the non-root refusal.
|
|
|
|
Prepare the platform image once, since the base images carry no Perl:
|
|
|
|
```sh
|
|
podman run --name sglang-prep registry.fedoraproject.org/fedora:44 dnf install -y perl
|
|
podman commit sglang-prep localhost/sglang-rig:fedora
|
|
podman rm sglang-prep
|
|
```
|
|
|
|
Then run the scenarios. The repository is mounted read-only at its own path and the
|
|
work directory is a scratch directory outside it:
|
|
|
|
```sh
|
|
podman run --rm --privileged \
|
|
-v "$PWD:$PWD:ro,z" \
|
|
-v /tmp/sglang-rig:/tmp/sglang-rig:Z \
|
|
localhost/sglang-rig:fedora perl "$PWD/tests/container/rig.pl"
|
|
```
|
|
|
|
The device nodes the script requires are faked by the rig itself, which is why the
|
|
container needs `--privileged`. The one scenario that checks the *missing* driver needs
|
|
a run without it:
|
|
|
|
```sh
|
|
podman run --rm \
|
|
-v "$PWD:$PWD:ro,z" \
|
|
-v /tmp/sglang-rig:/tmp/sglang-rig:Z \
|
|
localhost/sglang-rig:fedora perl "$PWD/tests/container/rig.pl" no_driver
|
|
```
|
|
|
|
The same rig was run on all three platforms, which is how the script's behaviour
|
|
against their package managers and their `os-release` was confirmed; only the image tag
|
|
in the commands changes:
|
|
|
|
| Platform | Image | Perl |
|
|
|---|---|---|
|
|
| Fedora 44 | `registry.fedoraproject.org/fedora:44` | 5.42.3 |
|
|
| CentOS Stream 10 | `quay.io/centos/centos:stream10` | 5.40.2 |
|
|
| openEuler 24.03 LTS | `docker.io/openeuler/openeuler:24.03-lts` | 5.38.0 |
|
|
|
|
A run leaves its log in `/tmp/sglang-rig/log/`, including `run.out`, the script's own
|
|
output, and `stubs.log`, every command it issued with its arguments. That log is the
|
|
quickest way to see what a run actually did.
|
|
|
|
## Running one script for real
|
|
|
|
The diagnostic scripts need no root and can simply be run:
|
|
|
|
```sh
|
|
perl network-diag.pl
|
|
perl system-diag.pl --json
|
|
```
|
|
|
|
The four that change a system need root and, except for
|
|
`workstation-setup.pl`, run in a disposable container the same way the rig does. Start
|
|
from the platform image, and read the summary before anything else: every one of them
|
|
supports `--dry-run`, which changes nothing and prints what it would do.
|
|
|
|
## Continuous integration
|
|
|
|
Workflows live in `.gitea/workflows/` and run on the project's own runners:
|
|
|
|
| Workflow | Trigger | Steps |
|
|
|---|---|---|
|
|
| Test | push or pull request to `development` | install Perl, then the syntax gate, the licence-header gate, the punctuation gate, and the six check files |
|
|
| Deploy | push to `main`, or dispatched by hand | write `SHA256SUMS`, pin the host key from `DEPLOY_KNOWN_HOSTS`, publish over rsync |
|
|
|
|
The container rig is deliberately not in the pipeline: it needs `--privileged`, and the
|
|
runner is a small box shared with the forge. A change that touches a system is expected
|
|
to be verified with the rig locally, and the pull request says which platform was used.
|
|
|
|
## Releases
|
|
|
|
There are none. Each script carries its own version string, and the deploy pipeline
|
|
publishes the scripts on every push to `main`, so a merge is a release and the commit
|
|
history is the record of what changed.
|