Compare commits

...
3 Commits
Author SHA1 Message Date
petrbalvin 108a166ee0 feat(server-setup): replace nginx with caddy
Test / test (push) Successful in 1m14s
Assisted-by: GLM 5.3 Flash
2026-09-29 00:32:44 +02:00
petrbalvin 37e06d090c docs: describe the sglang endpoint behind caddy
Assisted-by: GLM 5.3 Flash
2026-09-29 00:18:15 +02:00
petrbalvin 9455c65f10 feat(sglang-deploy): replace nginx with caddy
Assisted-by: GLM 5.3 Flash
2026-09-29 00:18:09 +02:00
12 changed files with 918 additions and 218 deletions
+3 -3
View File
@@ -22,7 +22,7 @@ own builtins, and none of them needs a module installed beside it.
automatic updates); `workstation-setup.pl` sets up a Fedora desktop (Brave, the automatic updates); `workstation-setup.pl` sets up a Fedora desktop (Brave, the
official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux). official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux).
- **Deployment**: `sglang-deploy.pl` puts an SGLang inference server on an AMD GPU - **Deployment**: `sglang-deploy.pl` puts an SGLang inference server on an AMD GPU
behind nginx with HTTPS and an API key, runs the engine from the project's ROCm behind Caddy with HTTPS and an API key, runs the engine from the project's ROCm
container image, and downloads the model weights from ModelScope. container image, and downloads the model weights from ModelScope.
- **Maintenance**: `system-optimise.pl` removes old kernels (the running one and one - **Maintenance**: `system-optimise.pl` removes old kernels (the running one and one
fallback always stay), vacuums journals, clears temporary files and core dumps, and fallback always stay), vacuums journals, clears temporary files and core dumps, and
@@ -90,9 +90,9 @@ a change would do without doing it.
| `network-diag.pl` | 2.0.0 | Network latency, DNS, MTU, packet loss and dual-stack on Linux and FreeBSD | | `network-diag.pl` | 2.0.0 | Network latency, DNS, MTU, packet loss and dual-stack on Linux and FreeBSD |
| `system-diag.pl` | 2.0.0 | System health with a grade from A to F. Linux only | | `system-diag.pl` | 2.0.0 | System health with a grade from A to F. Linux only |
| `security-audit.pl` | 2.0.0 | Security posture with a grade from A to F and an exit code for cron. Linux only | | `security-audit.pl` | 2.0.0 | Security posture with a grade from A to F and an exit code for cron. Linux only |
| `server-setup.pl` | 2.0.0 | Server initial setup for Fedora, CentOS Stream and openEuler | | `server-setup.pl` | 2.1.0 | Server initial setup for Fedora, CentOS Stream and openEuler |
| `workstation-setup.pl` | 2.0.0 | Fedora desktop setup | | `workstation-setup.pl` | 2.0.0 | Fedora desktop setup |
| `sglang-deploy.pl` | 2.0.0 | SGLang inference server on an AMD GPU, behind nginx with HTTPS | | `sglang-deploy.pl` | 2.1.0 | SGLang inference server on an AMD GPU, behind Caddy with HTTPS |
| `system-optimise.pl` | 2.0.0 | System cleanup; refuses rpm-ostree systems | | `system-optimise.pl` | 2.0.0 | System cleanup; refuses rpm-ostree systems |
The full flag reference is in [docs/CLI.md](docs/CLI.md). The full flag reference is in [docs/CLI.md](docs/CLI.md).
+1 -1
View File
@@ -61,7 +61,7 @@ is a defect worth reporting:
user who runs the script. user who runs the script.
- Missing hardening with no demonstrated impact, such as a script not setting a stricter - Missing hardening with no demonstrated impact, such as a script not setting a stricter
umask than the system's. umask than the system's.
- Flaws in a third-party tool the scripts drive (`dnf`, `podman`, `openssl`, `nginx` and - Flaws in a third-party tool the scripts drive (`dnf`, `podman`, `openssl`, `caddy` and
the rest): report those to that project, and here only when this repository's use of the rest): report those to that project, and here only when this repository's use of
the tool makes the flaw reachable in a way the tool's own documentation does not the tool makes the flaw reachable in a way the tool's own documentation does not
anticipate. anticipate.
+15 -3
View File
@@ -37,7 +37,7 @@ reports everything as already in place and changes nothing.
| `system-diag.pl` | Reading CPU, memory, disk, network, GPU, services, security and performance, and grading the result | Change anything on the host; it is read-only | | `system-diag.pl` | Reading CPU, memory, disk, network, GPU, services, security and performance, and grading the result | Change anything on the host; it is read-only |
| `server-setup.pl` | Base packages, firewall (with the SSH rule verified before the service starts), SELinux, Podman, automatic updates | Install the services themselves; that is the operator's, and the deploy scripts' | | `server-setup.pl` | Base packages, firewall (with the SSH rule verified before the service starts), SELinux, Podman, automatic updates | Install the services themselves; that is the operator's, and the deploy scripts' |
| `workstation-setup.pl` | A Fedora desktop: Brave, the official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux | Anything on a server distribution; it targets Fedora Workstation | | `workstation-setup.pl` | A Fedora desktop: Brave, the official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux | Anything on a server distribution; it targets Fedora Workstation |
| `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, nginx with TLS, the API key, the firewall rule and the SELinux boolean | The host's own setup, which `server-setup.pl` does first | | `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, Caddy with TLS, the API key, the firewall rule and the SELinux checks | The host's own setup, which `server-setup.pl` does first |
| `system-optimise.pl` | Old kernels (the running one and one fallback always stay), journals, temporary files, core dumps, and the package audit | Touch an rpm-ostree system, which it refuses | | `system-optimise.pl` | Old kernels (the running one and one fallback always stay), journals, temporary files, core dumps, and the package audit | Touch an rpm-ostree system, which it refuses |
## Data flow: a forked section collection ## Data flow: a forked section collection
@@ -97,8 +97,8 @@ that CentOS Stream and openEuler cannot carry at all.
```mermaid ```mermaid
flowchart LR flowchart LR
Client[client] -->|443| Nginx[nginx on the host, TLS] Client[client] -->|443| Caddy[Caddy on the host, TLS]
Nginx -->|loopback, plain HTTP| Engine[SGLang in a Podman container] Caddy -->|loopback, plain HTTP| Engine[SGLang in a Podman container]
Engine -->|device nodes| GPU[/dev/kfd, /dev/dri] Engine -->|device nodes| GPU[/dev/kfd, /dev/dri]
Engine -->|bind mount| Cache[state directory, model cache] Engine -->|bind mount| Cache[state directory, model cache]
Unit[systemd unit] -->|podman run| Engine Unit[systemd unit] -->|podman run| Engine
@@ -114,12 +114,24 @@ instead, and `--image` pins one. Radeon cards need `SGLANG_USE_AITER=false` and
`SGLANG_ROCM_FUSED_DECODE_MLA=false` in the unit, which the script writes for them and `SGLANG_ROCM_FUSED_DECODE_MLA=false` in the unit, which the script writes for them and
never for an Instinct host. never for an Instinct host.
Caddy serves the endpoint from a drop-in under `/etc/caddy/Caddyfile.d`, which the
distribution's default Caddyfile imports. Fedora carries the caddy package and CentOS
Stream gets it from EPEL, whose repository file the script installs first; openEuler
packages no caddy at all, so there the official release binary is installed instead,
with the unit file the package would have carried. The service runs as the caddy user,
so the private key is made group-readable for the caddy group, and on an
SELinux-enforcing host the packaged caddy runs unconfined: no boolean is needed. A
deployment made by an earlier release of this script carries an nginx configuration,
which both a deploy and an uninstall remove.
## Dependencies ## Dependencies
Nothing outside the interpreter, and nothing that has to be installed beyond the tools Nothing outside the interpreter, and nothing that has to be installed beyond the tools
each script's own dependency section installs. The non-obvious ones and their reasons: each script's own dependency section installs. The non-obvious ones and their reasons:
- `podman` for `sglang-deploy.pl`, because the engine is a container. - `podman` for `sglang-deploy.pl`, because the engine is a container.
- `caddy` for `sglang-deploy.pl`, because the endpoint serves TLS on 443; the
release binary and `tar` stand in where no repository carries the package.
- `lspci` for the GPU family, and `rocm-smi` in `system-diag.pl` for AMD memory and - `lspci` for the GPU family, and `rocm-smi` in `system-diag.pl` for AMD memory and
utilisation figures. utilisation figures.
- `sha256sum` in `workstation-setup.pl`, because Perl's builtins have no hash. - `sha256sum` in `workstation-setup.pl`, because Perl's builtins have no hash.
+5 -5
View File
@@ -111,7 +111,7 @@ verified by checksum; Brave's repository key is verified by fingerprint before i
``` ```
Usage: sglang-deploy.pl [options] Usage: sglang-deploy.pl [options]
--model ID Hugging Face model ID (menu when omitted) --model ID ModelScope model ID (menu when omitted)
--port N internal engine port (default: 8000, not 443) --port N internal engine port (default: 8000, not 443)
--tensor-parallel N GPUs for tensor parallelism (default: 1, written as --tensor-parallel N GPUs for tensor parallelism (default: 1, written as
the engine's --tp-size) the engine's --tp-size)
@@ -130,19 +130,19 @@ Usage: sglang-deploy.pl [options]
--api-key KEY API key for the endpoint (default: generate and --api-key KEY API key for the endpoint (default: generate and
store in /etc/sysconfig) store in /etc/sysconfig)
--dry-run preview without making changes --dry-run preview without making changes
--uninstall tear down the service, container, nginx config --uninstall tear down the service, container, caddy drop-in
and certificates and certificates
--help show this help --help show this help
--version show the version --version show the version
``` ```
Needs root. Without `--model` an interactive menu offers GLM 5.3, GLM 5.3 Flash, Needs root. Without `--model` an interactive menu offers GLM 5.3, GLM 5.3 Flash,
DeepSeek V4 Pro, DeepSeek V4 Flash, MiMo V2.5 Pro and MiMo V2.5, plus a free-form Qwen 3.8 Max, Qwen 3.8 Flash and DeepSeek V4.1 Flash, plus a free-form entry.
entry. `--tensor-parallel`, `--max-model-len` and `--gpu-memory-utilization` are `--tensor-parallel`, `--max-model-len` and `--gpu-memory-utilization` are
written to the unit as the engine's own `--tp-size`, `--context-length` and written to the unit as the engine's own `--tp-size`, `--context-length` and
`--mem-fraction-static`. The API key is shown once when it is generated; a key given `--mem-fraction-static`. The API key is shown once when it is generated; a key given
with `--api-key` is never echoed. `--uninstall` stops and disables the service, removes with `--api-key` is never echoed. `--uninstall` stops and disables the service, removes
the unit, the container, the nginx configuration and the certificates, and keeps the the unit, the container, the caddy drop-in and the certificates, and keeps the
image and the model cache. image and the model cache.
## system-optimise.pl ## system-optimise.pl
+3 -3
View File
@@ -84,7 +84,7 @@ deployment an operator cares about, so the pipeline that publishes them never ru
| Script | What it leaves behind | Where it is documented | | Script | What it leaves behind | Where it is documented |
|---|---|---| |---|---|---|
| `server-setup.pl` | Packages, firewalld with the SSH rule already allowed, SELinux enforcing, Podman, unattended updates | [docs/CLI.md](CLI.md) | | `server-setup.pl` | Packages, firewalld with the SSH rule already allowed, SELinux enforcing, Podman, unattended updates | [docs/CLI.md](CLI.md) |
| `sglang-deploy.pl` | A systemd unit running the engine in a Podman container, nginx with TLS in front, an API key file, the firewall rule and the SELinux boolean | [docs/ARCHITECTURE.md](ARCHITECTURE.md) | | `sglang-deploy.pl` | A systemd unit running the engine in a Podman container, Caddy with TLS in front, an API key file, the firewall rule and the SELinux checks | [docs/ARCHITECTURE.md](ARCHITECTURE.md) |
| `workstation-setup.pl` | A Fedora desktop with the toolchains and applications installed | [docs/CLI.md](CLI.md) | | `workstation-setup.pl` | A Fedora desktop with the toolchains and applications installed | [docs/CLI.md](CLI.md) |
The SGLang deployment is the only service in the collection, and the host needs no ROCm The SGLang deployment is the only service in the collection, and the host needs no ROCm
@@ -100,6 +100,6 @@ where the script reads them:
- The published copies are unversioned: whatever is on `main` is what is served. - The published copies are unversioned: whatever is on `main` is what is served.
- `sglang-deploy.pl` keeps the engine's API key in `/etc/sysconfig/sglang`, mode 0600, - `sglang-deploy.pl` keeps the engine's API key in `/etc/sysconfig/sglang`, mode 0600,
written by the script and never committed. A Hugging Face token for a gated model goes written by the script and never committed. A ModelScope token for a gated model goes
into that same file as `HF_TOKEN`, which the unit forwards to the container. into that same file as `MODELSCOPE_TOKEN`, which the unit forwards to the container.
- The deploy secrets live in the Gitea repository settings, under Actions, Secrets. - The deploy secrets live in the Gitea repository settings, under Actions, Secrets.
+9 -7
View File
@@ -61,13 +61,15 @@ only be exercised as root.
`tests/container/rig.pl` runs the real `sglang-deploy.pl` as root inside a container `tests/container/rig.pl` runs the real `sglang-deploy.pl` as root inside a container
against a stub `PATH`: every command the script drives (`dnf`, `rpm`, `podman`, against a stub `PATH`: every command the script drives (`dnf`, `rpm`, `podman`,
`systemctl`, `curl`, `openssl`, `nginx`, `firewall-cmd`, `getsebool`, `lspci`) is a `systemctl`, `curl`, `openssl`, `caddy`, `tar`, `useradd`, `semodule`, `firewall-cmd`,
stub that answers from a fixture, so a run is deterministic and needs no network and no `getsebool`, `lspci`) is a stub that answers from a fixture, so a run is deterministic
GPU. It covers the deploy end to end: the resolved image tag, the unit file, the nginx and needs no network and no GPU. It covers the deploy end to end: the resolved image
configuration, the TLS certificate and its modes, the API key file, the SELinux tag, the unit file, the caddy drop-in and the main Caddyfile's import, the release
boolean, the firewall rule, the idempotent second run, the dry run, the uninstall, the binary install for the hosts without a caddy package, the EPEL bootstrap on CentOS
Radeon and MI300 paths, the offline and unpublished-tag failures, the argument Stream, the legacy nginx clean-up, the TLS certificate and its modes, the API key file,
validation and the non-root refusal. the SELinux decisions, the firewall rule, the idempotent second run, the dry run, the
uninstall, the Radeon and MI300 paths, the offline and unpublished-tag failures, the
argument validation and the non-root refusal.
Prepare the platform image once, since the base images carry no Perl: Prepare the platform image once, since the base images carry no Perl:
+43 -10
View File
@@ -44,7 +44,7 @@
use strict; use strict;
use warnings; use warnings;
my $VERSION = '2.0.0'; my $VERSION = '2.1.0';
my $BOLD = "\033[1m"; my $BOLD = "\033[1m";
my $RED = "\033[31m"; my $RED = "\033[31m";
@@ -67,16 +67,39 @@ my %SUPPORTED_OS = (
); );
# Base packages installed on Fedora, CentOS Stream and openEuler alike. # Base packages installed on Fedora, CentOS Stream and openEuler alike.
# caddy is the reverse proxy the deploy scripts serve the endpoints through;
# openEuler ships no package for it, and install_packages says so and leaves
# it to the script that needs it.
my @BASE_PACKAGES = qw( my @BASE_PACKAGES = qw(
nano curl wget htop tmux rsync nginx openssl jq fastfetch nano curl wget htop tmux rsync caddy openssl jq fastfetch
); );
# Services to open in firewalld by default. ssh is mandatory: losing it means # Services to open in firewalld by default. ssh is mandatory: losing it means
# locking out remote administration. # locking out remote administration.
my @FIREWALL_SERVICES = ('ssh'); my @FIREWALL_SERVICES = ('ssh');
# Opened in firewalld only when nginx is installed, which it is by default. # Opened in firewalld only when caddy is installed, which it is by default
my @NGINX_FIREWALL_SERVICES = ('http', 'https'); # wherever the package exists.
my @CADDY_FIREWALL_SERVICES = ('http', 'https');
# The one base package a distribution's repositories do not carry, with the
# reason and who provides it instead.
my %UNPACKAGED = (
openeuler => {
caddy => 'openEuler ships no caddy package; the scripts that need the '
. 'proxy install the release binary themselves',
},
);
# Test-visible accessors: the catalogue is lexical to this file, so the checks
# under tests/ read the base package list and the unpackaged map through these.
sub base_packages { return @BASE_PACKAGES; }
sub unpackaged_for {
my ($os_id) = @_;
return () unless exists $UNPACKAGED{$os_id};
return %{ $UNPACKAGED{$os_id} };
}
# dnf-automatic configuration file. dnf 4 ships it with defaults; dnf 5 reads host # dnf-automatic configuration file. dnf 4 ships it with defaults; dnf 5 reads host
# overrides from this path (its own defaults live in /usr/share/dnf5). # overrides from this path (its own defaults live in /usr/share/dnf5).
@@ -565,19 +588,29 @@ sub ensure_epel {
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
sub install_packages { sub install_packages {
my ($dry_run) = @_; my ($os_id, $dry_run) = @_;
my (@installed, @skipped, @failed, @to_install); my (@installed, @skipped, @failed, @to_install, @unpackaged);
_status('Checking base packages'); _status('Checking base packages');
for my $pkg (@BASE_PACKAGES) { for my $pkg (@BASE_PACKAGES) {
# exists and not a bare lookup: descending into a missing os key
# would autovivify it.
if (exists $UNPACKAGED{$os_id} && exists $UNPACKAGED{$os_id}{$pkg}) {
push @unpackaged, [$pkg, $UNPACKAGED{$os_id}{$pkg}];
next;
}
if (rpm_installed($pkg)) { push @skipped, $pkg } if (rpm_installed($pkg)) { push @skipped, $pkg }
else { push @to_install, $pkg } else { push @to_install, $pkg }
} }
my $already = scalar @skipped; my $already = scalar @skipped;
my $missing = scalar @to_install; my $missing = scalar @to_install;
my $total = scalar @BASE_PACKAGES; my $total = scalar @BASE_PACKAGES - scalar @unpackaged;
_status_done("$already/$total already installed"); _status_done("$already/$total already installed");
for my $entry (@unpackaged) {
my ($pkg, $reason) = @$entry;
_info("$pkg is not packaged on this distribution: $reason");
}
if (!@to_install) { if (!@to_install) {
_ok('All base packages already present'); _ok('All base packages already present');
@@ -663,9 +696,9 @@ sub setup_firewall {
$info{installed} = 1; $info{installed} = 1;
} }
# ssh is mandatory; http and https only when nginx is present. # ssh is mandatory; http and https only when caddy is present.
my @services = @FIREWALL_SERVICES; my @services = @FIREWALL_SERVICES;
push @services, @NGINX_FIREWALL_SERVICES if rpm_installed('nginx'); push @services, @CADDY_FIREWALL_SERVICES if rpm_installed('caddy');
# --- Permanent rules first, through the offline client, which needs no daemon # --- Permanent rules first, through the offline client, which needs no daemon
my $permanent_changed = 0; my $permanent_changed = 0;
@@ -1776,7 +1809,7 @@ sub main {
# 3. Base packages # 3. Base packages
print STDERR "\n${BOLD}── Base Packages ──$RESET\n"; print STDERR "\n${BOLD}── Base Packages ──$RESET\n";
if (!$opt{skip_packages}) { if (!$opt{skip_packages}) {
$results{packages} = install_packages($opt{dry_run}); $results{packages} = install_packages($os_id, $opt{dry_run});
if (@{ $results{packages}{failed} }) { if (@{ $results{packages}{failed} }) {
push @warnings, 'Some packages failed to install: ' push @warnings, 'Some packages failed to install: '
. join(', ', @{ $results{packages}{failed} }); . join(', ', @{ $results{packages}{failed} });
+509 -126
View File
@@ -4,12 +4,14 @@
# Idempotent SGLang deployment for AMD ROCm GPUs. # Idempotent SGLang deployment for AMD ROCm GPUs.
# #
# SGLang behind nginx with self-signed TLS on Fedora, CentOS Stream or openEuler. # SGLang behind Caddy with self-signed TLS on Fedora, CentOS Stream or openEuler.
# The engine binds to ::1 (IPv4 loopback fallback) on port 8000, internal only; # The engine binds to ::1 (IPv4 loopback fallback) on port 8000, internal only;
# nginx proxies :443 to the loopback upstream with streaming (SSE) support. The # Caddy proxies :443 to the loopback upstream and streams (SSE) unbuffered. The
# endpoint requires an API key, delivered to the service through a 0600 # endpoint requires an API key, delivered to the service through a 0600
# EnvironmentFile; nginx to engine proxying is allowed through SELinux on enforcing # EnvironmentFile. Caddy's service runs as the caddy user, so the private key is
# systems. # made group-readable for the caddy group, and on SELinux-enforcing hosts the
# distribution's caddy runs unconfined, which needs no boolean; where a confined
# caddy policy is loaded anyway the script sets httpd_can_network_connect.
# #
# Why the engine runs in a container rather than straight on the host: # Why the engine runs in a container rather than straight on the host:
# #
@@ -19,8 +21,8 @@
# Rust), which Fedora carries only partly and which CentOS Stream and openEuler, where # Rust), which Fedora carries only partly and which CentOS Stream and openEuler, where
# ROCm itself is unsupported by AMD, cannot carry at all. Both AMD and SGLang document # ROCm itself is unsupported by AMD, cannot carry at all. Both AMD and SGLang document
# the container as the way to run SGLang on ROCm, so the container is what this script # the container as the way to run SGLang on ROCm, so the container is what this script
# deploys: podman runs the official image, and the host keeps nginx, TLS, the API key, # deploys: podman runs the official image, and the host keeps Caddy, TLS, the API key,
# the firewall and the SELinux boolean. The host needs no ROCm userland, only the # the firewall and the SELinux story. The host needs no ROCm userland, only the
# amdgpu kernel driver and its device nodes, /dev/kfd and /dev/dri. # amdgpu kernel driver and its device nodes, /dev/kfd and /dev/dri.
# #
# Radeon cards: the project publishes no stable image for gfx1151 (Strix Halo, the # Radeon cards: the project publishes no stable image for gfx1151 (Strix Halo, the
@@ -42,7 +44,9 @@
# keeps stdout and stderr apart in the scratch directory. # keeps stdout and stderr apart in the scratch directory.
# #
# External binaries used: dnf, rpm, curl, podman, lspci, openssl, systemctl, # External binaries used: dnf, rpm, curl, podman, lspci, openssl, systemctl,
# getenforce, getsebool, setsebool, firewall-cmd and nginx. # getenforce, getsebool, setsebool, semodule, firewall-cmd, caddy, tar, install and
# useradd (the last four only on a host where no repository carries the caddy
# package and the release binary is installed instead).
# #
# Usage: # Usage:
# sglang-deploy.pl # interactive model selection # sglang-deploy.pl # interactive model selection
@@ -55,7 +59,7 @@
use strict; use strict;
use warnings; use warnings;
my $VERSION = '2.0.0'; my $VERSION = '2.1.0';
my $BOLD = "\033[1m"; my $BOLD = "\033[1m";
my $RED = "\033[31m"; my $RED = "\033[31m";
@@ -78,11 +82,11 @@ my %SUPPORTED_OS = (
); );
# Tools this script itself needs. podman is the engine's runtime: SGLang ships no # Tools this script itself needs. podman is the engine's runtime: SGLang ships no
# ROCm wheel, so the server runs from the project's own ROCm image. # ROCm wheel, so the server runs from the project's own ROCm image. caddy is
my @DNF_PACKAGES = qw(pciutils curl openssl podman); # installed in its own step rather than in this transaction: a name the
# repositories do not carry aborts the whole dnf run, and openEuler ships no
# Packages the engine expects from server-setup.pl (warning only, not installed here). # caddy at all (there the release binary takes its place).
my @REQUIRED_SERVER_PACKAGES = (['nginx', 'reverse proxy']); my @DNF_PACKAGES = qw(pciutils curl openssl podman tar);
# Default models for interactive selection when --model is omitted, keyed by # Default models for interactive selection when --model is omitted, keyed by
# display name. Every ID is a ModelScope repository, which is where the engine # display name. Every ID is a ModelScope repository, which is where the engine
@@ -176,6 +180,19 @@ my $DEFAULT_CERT_DIR = '/etc/ssl/sglang';
my $DEFAULT_SERVICE = 'sglang'; my $DEFAULT_SERVICE = 'sglang';
my $INTERNAL_PORT = 8000; my $INTERNAL_PORT = 8000;
# Caddy, the endpoint's TLS proxy. Fedora carries the package, CentOS Stream
# gets it from EPEL, and openEuler ships none, so where no package can be
# installed the official release binary takes its place, with the unit file the
# package would have carried. The distribution's default Caddyfile imports the
# Caddyfile.d directory, which is the drop-in this script writes; a main file
# without the import line gets it appended.
my $CADDY_RELEASES_API = 'https://api.github.com/repos/caddyserver/caddy/releases/latest';
my $CADDY_FALLBACK_VERSION = '2.10.2';
my $CADDY_BINARY_PATH = '/usr/local/bin/caddy';
my $CADDY_CONFIG_DIR = '/etc/caddy';
my $CADDY_IMPORT_LINE = 'import Caddyfile.d/*.caddyfile';
my $LEGACY_NGINX_DIR = '/etc/nginx/conf.d';
# ModelScope model IDs look like "org/name", the same shape Hugging Face uses. # ModelScope model IDs look like "org/name", the same shape Hugging Face uses.
# The strict pattern also keeps systemd specifier characters (%) and whitespace # The strict pattern also keeps systemd specifier characters (%) and whitespace
# out of unit files. # out of unit files.
@@ -287,7 +304,7 @@ sub write_file {
my ($path, $content) = @_; my ($path, $content) = @_;
open(my $fh, '>', $path) or return 0; open(my $fh, '>', $path) or return 0;
# The flush of a buffered handle surfaces at close, so close is checked too: # The flush of a buffered handle surfaces at close, so close is checked too:
# a full disk must not report a truncated unit file or nginx configuration # a full disk must not report a truncated unit file or Caddyfile drop-in
# as written. # as written.
my $ok = print {$fh} $content; my $ok = print {$fh} $content;
$ok = 0 unless close($fh); $ok = 0 unless close($fh);
@@ -691,7 +708,7 @@ sub ms_model_status {
return 'unknown'; return 'unknown';
} }
# Return (bind_host, nginx_upstream_host): IPv6 ::1 first. # Return (bind_host, caddy_upstream_host): IPv6 ::1 first.
# #
# Falls back to 127.0.0.1 on kernels with IPv6 disabled # Falls back to 127.0.0.1 on kernels with IPv6 disabled
# (net.ipv6.conf.all.disable_ipv6=1), where binding ::1 would fail. # (net.ipv6.conf.all.disable_ipv6=1), where binding ::1 would fail.
@@ -760,7 +777,7 @@ sub cert_san {
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
sub install_system_deps { sub install_system_deps {
my ($dry_run) = @_; my ($os_id, $dry_run) = @_;
my %results = (installed => [], skipped => [], failed => []); my %results = (installed => [], skipped => [], failed => []);
my @missing; my @missing;
@@ -820,17 +837,199 @@ sub install_system_deps {
_fail('podman is not installed: the engine runs as a container and cannot start'); _fail('podman is not installed: the engine runs as a container and cannot start');
} }
# Packages expected from server-setup.pl (warning only, not installed here). # Caddy proxies the endpoint on 443. Fedora carries the package; CentOS
for my $entry (@REQUIRED_SERVER_PACKAGES) { # Stream carries it in EPEL, whose repository file installs first; where no
my ($pkg, $purpose) = @$entry; # repository carries it at all (openEuler ships none), the official release
if (!rpm_installed($pkg)) { # binary takes its place, unit file included. The step is separate from the
_warn("$pkg ($purpose) is not installed: run server-setup.pl first"); # transaction above, because one unresolvable name aborts a whole dnf run.
my $caddy = caddy_binary();
if (defined $caddy) {
_status('Checking caddy');
my $result = run([$caddy, 'version'], timeout => 30);
my $version = $result->{out};
$version =~ s/^\s+//;
$version =~ s/\s+$//;
$version = (split /\s+/, $version)[0] // '';
_status_done($version ne '' ? $version : 'installed');
$results{caddy} = $version ne '' ? $version : 'installed';
}
elsif ($dry_run) {
_status('Checking caddy');
_status_done('would install');
$results{caddy} = 'dry run';
}
else {
# CentOS Stream carries caddy in EPEL, and the repository file ships in
# its extras repository: installing it first is what makes caddy
# resolvable in the transaction below.
if ($os_id eq 'centos' && !rpm_installed('epel-release')) {
_status('Enabling EPEL (caddy is packaged there)');
my $epel = run(['dnf', 'install', '-y', 'epel-release'], timeout => $DNF_TIMEOUT);
if ($epel->{rc} == 0) {
_status_done('ok');
}
else {
_status_done('failed');
my $err = $epel->{err};
$err =~ s/\s+$//;
_info("dnf stderr: $err") if length $err;
}
}
_status('Installing caddy');
my $package = run(['dnf', 'install', '-y', 'caddy'], timeout => $DNF_TIMEOUT);
if ($package->{rc} == 0) {
_status_done('package');
$results{caddy} = 'package';
}
elsif (install_caddy_binary()) {
_status_done('release binary');
$results{caddy} = 'release binary';
}
else {
_status_done('failed');
$results{caddy} = undef;
_fail('caddy is not available: the endpoint cannot be served on 443');
} }
} }
return \%results; return \%results;
} }
# The caddy binary the script drives, package or release binary. An absolute
# fallback is needed because a systemd unit and a fresh install reach the
# binary before any PATH that carries /usr/local/bin.
sub caddy_binary {
my $exe = find_exe('caddy');
return $exe if defined $exe;
return (-f $CADDY_BINARY_PATH && -x _) ? $CADDY_BINARY_PATH : undef;
}
# The newest caddy release version ('2.10.2'), from the GitHub API with a
# constant as the fallback. The charset bound keeps whatever the API answers
# out of the download URL.
sub resolve_caddy_version {
my $listing = fetch_text($CADDY_RELEASES_API);
if ($listing =~ /"tag_name"\s*:\s*"v(\d+\.\d+\.\d+)"/) {
return $1;
}
return $CADDY_FALLBACK_VERSION;
}
# caddy publishes release assets as caddy_VERSION_linux_ARCH.tar.gz.
sub uname_to_arch {
my ($machine) = @_;
return 'amd64' if $machine eq 'x86_64';
return 'arm64' if $machine eq 'aarch64';
return undef;
}
sub caddy_asset_url {
my ($version, $arch) = @_;
return "https://github.com/caddyserver/caddy/releases/download"
. "/v$version/caddy_${version}_linux_${arch}.tar.gz";
}
# Install caddy from the official release binary, for the hosts no repository
# carries the package for (openEuler ships none). Everything the package would
# have provided is provided here: the binary, the directories, the service user
# and the unit file, copied from the distribution's own unit. The caller owns
# the progress line; this reports only failures.
sub install_caddy_binary {
my $version = resolve_caddy_version();
my $machine = run(['uname', '-m'], timeout => 15)->{out};
$machine =~ s/^\s+//;
$machine =~ s/\s+$//;
my $arch = uname_to_arch($machine);
if (!defined $arch) {
_fail("caddy publishes no release binary for $machine");
return 0;
}
my $curl = find_exe('curl');
my $tar = find_exe('tar');
my $install = find_exe('install');
if (!defined $curl || !defined $tar || !defined $install) {
_fail('curl, tar or install is missing: cannot install the caddy release binary');
return 0;
}
my $dir = scratch_dir();
my $tarball = "$dir/caddy.tar.gz";
my $download = run([$curl, '-fsSL', '-o', $tarball, caddy_asset_url($version, $arch)],
timeout => 300);
if ($download->{rc} != 0) {
my $err = $download->{err};
$err =~ s/\s+$//;
_fail("The caddy release download failed: $err");
return 0;
}
my $extract = "$dir/caddy-extract";
mkdir($extract, 0700);
my $unpacked = run([$tar, '-xzf', $tarball, '-C', $extract], timeout => 60);
if ($unpacked->{rc} != 0 || !-f "$extract/caddy") {
_fail('The caddy release archive is not readable');
return 0;
}
for my $path ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d", '/var/lib/caddy') {
mkdir($path, 0755) unless -d $path;
}
my $placed = run([$install, '-m', '0755', "$extract/caddy", $CADDY_BINARY_PATH],
timeout => 30);
if ($placed->{rc} != 0) {
my $err = $placed->{err};
$err =~ s/\s+$//;
_fail("Could not install $CADDY_BINARY_PATH: $err");
return 0;
}
if (!getpwnam('caddy')) {
my $user = run(['useradd', '--system', '--home-dir', '/var/lib/caddy',
'--create-home', '--shell', '/sbin/nologin', 'caddy'], timeout => 30);
if ($user->{rc} != 0) {
_warn('The caddy user could not be created: create it before starting caddy');
}
}
my $unit_path = '/etc/systemd/system/caddy.service';
if (!write_file($unit_path, caddy_unit_content())) {
_fail("Could not write $unit_path: " . os_error_text($unit_path));
return 0;
}
run(['systemctl', 'daemon-reload'], timeout => 30);
return 1;
}
# The unit file for a release-binary install: the distribution's own unit, with
# the binary path adjusted. validate in ExecStartPre is what keeps a broken
# Caddyfile from taking the service down at boot.
sub caddy_unit_content {
return <<"UNIT";
[Unit]
Description=Caddy web server
Documentation=https://caddyserver.com/docs/
After=network.target
[Service]
User=caddy
Group=caddy
ExecStartPre=$CADDY_BINARY_PATH validate --config $CADDY_CONFIG_DIR/Caddyfile
ExecStart=$CADDY_BINARY_PATH run --environ --config $CADDY_CONFIG_DIR/Caddyfile
ExecReload=$CADDY_BINARY_PATH reload --config $CADDY_CONFIG_DIR/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectHome=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
[Install]
WantedBy=multi-user.target
UNIT
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 2. Pre-flight checks # 2. Pre-flight checks
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -1119,6 +1318,21 @@ sub fetch_engine_image {
# 5. TLS certificate (self-signed) # 5. TLS certificate (self-signed)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# The caddy service runs as the caddy user (the package creates it), so the
# private key has to cross the group line: root keeps the ownership and the
# caddy group gets read. Without the group (the package is missing) the key
# stays root-only and the caddy step reports what is missing.
sub ensure_caddy_key_readable {
my ($key_path) = @_;
my ($group, undef, $gid) = getgrnam('caddy');
if (!defined $group) {
return 0;
}
chown(0, $gid, $key_path);
chmod(0640, $key_path);
return 1;
}
sub setup_tls { sub setup_tls {
my ($cert_dir, $dry_run) = @_; my ($cert_dir, $dry_run) = @_;
my %results; my %results;
@@ -1127,6 +1341,7 @@ sub setup_tls {
_status('Checking TLS certificate'); _status('Checking TLS certificate');
if (-f $crt_path && -f $key_path) { if (-f $crt_path && -f $key_path) {
$results{key_readable} = ensure_caddy_key_readable($key_path);
_status_done('already exists'); _status_done('already exists');
$results{cert_exists} = 1; $results{cert_exists} = 1;
return \%results; return \%results;
@@ -1172,6 +1387,7 @@ sub setup_tls {
if ($result->{rc} == 0) { if ($result->{rc} == 0) {
chmod 0600, $key_path; chmod 0600, $key_path;
chmod 0644, $crt_path; chmod 0644, $crt_path;
$results{key_readable} = ensure_caddy_key_readable($key_path);
_status_done('generated'); _status_done('generated');
$results{cert_created} = 1; $results{cert_created} = 1;
} }
@@ -1288,10 +1504,13 @@ sub encode_base64url {
# 7. SELinux # 7. SELinux
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Allow nginx to reach the engine's port on SELinux-enforcing systems. # Allow Caddy to reach the engine's port on SELinux-enforcing systems.
# #
# http_port_t covers 80/81/443/488/8008/8009/8443/9000 but not the engine's port, so # The distributions package caddy without an SELinux policy module, so its
# on enforcing systems nginx needs httpd_can_network_connect. # service runs unconfined and needs no boolean at all. Where a confined caddy
# policy is loaded anyway (a local module), proxying to the engine's port needs
# httpd_can_network_connect: http_port_t covers 80/81/443/488/8008/8009/8443/9000
# but not the engine's port.
sub setup_selinux { sub setup_selinux {
my ($dry_run) = @_; my ($dry_run) = @_;
my %results; my %results;
@@ -1315,12 +1534,28 @@ sub setup_selinux {
} }
_status_done('enforcing'); _status_done('enforcing');
_status('Checking for a confined caddy policy');
my $semodule = find_exe('semodule');
my $confined = 0;
if (defined $semodule) {
my $list = run([$semodule, '-l'], timeout => 30);
# "semodule -l" lines read "100 caddy(pp)" or the plain "caddy 1.0"
# of older releases; the priority column is optional in the match.
$confined = 1 if $list->{rc} == 0 && $list->{out} =~ /^\s*(?:\d+\s+)?\S*caddy\b/m;
}
if (!$confined) {
_status_done('none (caddy runs unconfined)');
$results{selinux} = 'unconfined';
return \%results;
}
_status_done('confined policy loaded');
_status('Checking httpd_can_network_connect boolean'); _status('Checking httpd_can_network_connect boolean');
my $getsebool = find_exe('getsebool'); my $getsebool = find_exe('getsebool');
my $setsebool = find_exe('setsebool'); my $setsebool = find_exe('setsebool');
if (!defined $getsebool || !defined $setsebool) { if (!defined $getsebool || !defined $setsebool) {
_status_done('tools missing'); _status_done('tools missing');
_warn('getsebool/setsebool not found: nginx proxying may be blocked (502)'); _warn('getsebool/setsebool not found: caddy proxying may be blocked (502)');
$results{selinux} = 'failed'; $results{selinux} = 'failed';
return \%results; return \%results;
} }
@@ -1356,59 +1591,143 @@ sub setup_selinux {
} }
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 8. nginx configuration # 8. Caddy configuration
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Return the nginx server block content. # Return the Caddyfile drop-in for the endpoint.
# #
# HTTP/1.1 with an empty Connection header and proxy_buffering off are required for # Caddy streams proxied responses immediately when flush_interval is negative,
# the engine's SSE streaming: nginx's defaults (HTTP/1.0, buffering on) would hold a # which the engine's SSE completions need; the nginx equivalent was HTTP/1.1
# whole streamed completion until generation finishes. The IPv6 listener is emitted # with proxy_buffering off. Caddy sets X-Forwarded-For and X-Forwarded-Proto
# only when the kernel actually has IPv6 enabled (the same check as detect_loopback); # itself and passes the Host header through, so only X-Real-IP is written.
# socket() on [::]:443 would otherwise fail with EAFNOSUPPORT and take nginx down. # There is no read timeout: a completion that generates for minutes must not be
sub nginx_conf_content { # cut at a fixed limit, and the response ends when the engine ends it. No bind
# directive is written: Caddy listens on both loopback families on kernels with
# IPv6 and falls back to IPv4 alone where the kernel has none. The nesting is
# tab-indented, which is how the Caddyfile is formatted.
sub caddyfile_content {
my ($port, $upstream_host, $cert_dir) = @_; my ($port, $upstream_host, $cert_dir) = @_;
my $ipv6_listen = -e '/proc/net/if_inet6' ? "listen [::]:443 ssl;\n " : '';
return <<"CONF"; return <<"CONF";
server { :443 {
${ipv6_listen}listen 443 ssl; tls $cert_dir/$CONTAINER_NAME.crt $cert_dir/$CONTAINER_NAME.key
server_name _; request_body {
max_size 50MB
ssl_certificate $cert_dir/$CONTAINER_NAME.crt; }
ssl_certificate_key $cert_dir/$CONTAINER_NAME.key; reverse_proxy $upstream_host:$port {
ssl_protocols TLSv1.2 TLSv1.3; flush_interval -1
header_up X-Real-IP {remote_host}
client_max_body_size 50m;
location / {
proxy_pass http://$upstream_host:$port;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host \$host;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
proxy_buffering off;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
} }
} }
CONF CONF
} }
sub nginx_conf_path { sub caddyfile_path {
my ($service_name) = @_; my ($service_name) = @_;
return "/etc/nginx/conf.d/$service_name.conf"; return "$CADDY_CONFIG_DIR/Caddyfile.d/$service_name.caddyfile";
} }
sub setup_nginx { sub caddy_main_config {
return "$CADDY_CONFIG_DIR/Caddyfile";
}
# The main Caddyfile must import the drop-in directory. The distributions'
# default file carries the import; a host without the file gets a minimal one,
# and one that does not import gets the line appended, which is inert while the
# directory holds nothing else.
sub ensure_caddy_import {
my ($dry_run) = @_;
my %results;
my $main = caddy_main_config();
my $current = -f $main ? slurp($main) : '';
_status('Checking the Caddyfile import');
# Any import of the drop-in directory counts, not only this script's exact
# line: appending a second one would make Caddy read every drop-in twice.
if ($current =~ /^\s*import\s+Caddyfile\.d\//m) {
_status_done('present');
$results{caddy_import} = 'present';
return \%results;
}
if ($dry_run) {
_status_done('would add');
$results{caddy_import} = 'dry run';
return \%results;
}
for my $dir ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d") {
mkdir($dir, 0755) unless -d $dir;
}
my $desired = length($current)
? $current . (substr($current, -1) eq "\n" ? '' : "\n") . "$CADDY_IMPORT_LINE\n"
: "$CADDY_IMPORT_LINE\n";
if (write_file($main, $desired)) {
chmod 0644, $main;
_status_done(length $current ? 'added' : 'created');
$results{caddy_import} = length $current ? 'added' : 'created';
}
else {
_status_done('failed');
_fail("Could not write $main: " . os_error_text($main));
$results{caddy_import} = 0;
}
return \%results;
}
# A deployment made by the nginx release leaves its drop-in behind. Remove it,
# so exactly one proxy owns :443; nginx itself stays, for whatever else it serves.
sub remove_legacy_nginx {
my ($service_name, $dry_run) = @_;
my %results;
my $legacy = "$LEGACY_NGINX_DIR/$service_name.conf";
return \%results unless -f $legacy;
_status('Removing the legacy nginx configuration');
if ($dry_run) {
_status_done('dry run');
$results{legacy_nginx_removed} = 'dry run';
return \%results;
}
unlink($legacy);
if (systemctl_is_active('nginx')) {
my $reload = run(['systemctl', 'reload', 'nginx'], timeout => 30);
if ($reload->{rc} == 0) {
_status_done('removed and nginx reloaded');
}
else {
_status_done('removed (nginx reload failed)');
my $err = $reload->{err};
$err =~ s/\s+$//;
_warn("nginx reload failed: $err");
}
}
else {
_status_done('removed (nginx not running)');
}
$results{legacy_nginx_removed} = 1;
return \%results;
}
sub setup_caddy {
my ($port, $upstream_host, $cert_dir, $service_name, $dry_run) = @_; my ($port, $upstream_host, $cert_dir, $service_name, $dry_run) = @_;
my %results; my %results;
my $conf_path = nginx_conf_path($service_name);
my $desired_content = nginx_conf_content($port, $upstream_host, $cert_dir);
# Write the nginx config if it is missing or different. my $import = ensure_caddy_import($dry_run);
_status('Checking nginx configuration'); $results{caddy_import} = $import->{caddy_import};
my $legacy = remove_legacy_nginx($service_name, $dry_run);
$results{legacy_nginx_removed} = $legacy->{legacy_nginx_removed}
if defined $legacy->{legacy_nginx_removed};
my $conf_path = caddyfile_path($service_name);
my $desired_content = caddyfile_content($port, $upstream_host, $cert_dir);
if (!$dry_run) {
for my $dir ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d") {
mkdir($dir, 0755) unless -d $dir;
}
}
# Write the drop-in if it is missing or different.
_status('Checking the caddy configuration');
if (-f $conf_path) { if (-f $conf_path) {
my $current = slurp($conf_path); my $current = slurp($conf_path);
$current =~ s/^\s+//; $current =~ s/^\s+//;
@@ -1418,95 +1737,107 @@ sub setup_nginx {
$desired =~ s/\s+$//; $desired =~ s/\s+$//;
if ($current eq $desired) { if ($current eq $desired) {
_status_done('already configured'); _status_done('already configured');
$results{nginx_configured} = 1; $results{caddy_configured} = 1;
} }
elsif ($dry_run) { elsif ($dry_run) {
_status_done('would update'); _status_done('would update');
$results{nginx_configured} = 'dry run'; $results{caddy_configured} = 'dry run';
} }
elsif (write_file($conf_path, $desired_content)) { elsif (write_file($conf_path, $desired_content)) {
chmod 0644, $conf_path;
_status_done('updated'); _status_done('updated');
$results{nginx_configured} = 1; $results{caddy_configured} = 1;
} }
else { else {
_status_done('failed'); _status_done('failed');
_fail("Could not write $conf_path: " . os_error_text($conf_path)); _fail("Could not write $conf_path: " . os_error_text($conf_path));
$results{nginx_configured} = 0; $results{caddy_configured} = 0;
} }
} }
elsif ($dry_run) { elsif ($dry_run) {
_status_done('would create'); _status_done('would create');
$results{nginx_configured} = 'dry run'; $results{caddy_configured} = 'dry run';
} }
elsif (write_file($conf_path, $desired_content)) { elsif (write_file($conf_path, $desired_content)) {
chmod 0644, $conf_path;
_status_done('created'); _status_done('created');
$results{nginx_configured} = 1; $results{caddy_configured} = 1;
} }
else { else {
_status_done('failed'); _status_done('failed');
_fail("Could not write $conf_path: " . os_error_text($conf_path)); _fail("Could not write $conf_path: " . os_error_text($conf_path));
$results{nginx_configured} = 0; $results{caddy_configured} = 0;
} }
# Ensure nginx is enabled and running (handles the enabled-but-stopped case). # Ensure caddy is enabled and running (handles the enabled-but-stopped case).
_status('Ensuring nginx service is enabled and running'); _status('Ensuring caddy service is enabled and running');
my $nginx_running = systemctl_is_active('nginx'); my $caddy_running = systemctl_is_active('caddy');
if (systemctl_is_enabled('nginx') && $nginx_running) { if (systemctl_is_enabled('caddy') && $caddy_running) {
_status_done('running'); _status_done('running');
$results{nginx_running} = 1; $results{caddy_running} = 1;
} }
elsif ($dry_run) { elsif ($dry_run) {
_status_done('dry run'); _status_done('dry run');
$results{nginx_running} = 'dry run'; $results{caddy_running} = 'dry run';
} }
else { else {
my $start_result = systemctl_is_enabled('nginx') my $start_result = systemctl_is_enabled('caddy')
? run(['systemctl', 'start', 'nginx'], timeout => 30) ? run(['systemctl', 'start', 'caddy'], timeout => 30)
: run(['systemctl', 'enable', '--now', 'nginx'], timeout => 30); : run(['systemctl', 'enable', '--now', 'caddy'], timeout => 30);
if ($start_result->{rc} == 0) { if ($start_result->{rc} == 0) {
_status_done('enabled and started'); _status_done('enabled and started');
$results{nginx_running} = 1; $results{caddy_running} = 1;
} }
else { else {
_status_done('failed'); _status_done('failed');
my $err = $start_result->{err}; my $err = $start_result->{err};
$err =~ s/\s+$//; $err =~ s/\s+$//;
_warn("Could not start nginx: $err"); _warn("Could not start caddy: $err");
$results{nginx_running} = 0; $results{caddy_running} = 0;
} }
} }
# Test and reload the configuration (only possible when nginx is running). # Validate and reload the configuration (only possible when caddy is running).
_status('Reloading nginx configuration'); _status('Reloading caddy configuration');
if ($dry_run) { if ($dry_run) {
_status_done('dry run'); _status_done('dry run');
$results{nginx_reloaded} = 'dry run'; $results{caddy_reloaded} = 'dry run';
} }
elsif (!$results{nginx_running}) { elsif (!$results{caddy_running}) {
_status_done('skipped (nginx not running)'); _status_done('skipped (caddy not running)');
$results{nginx_reloaded} = 0; $results{caddy_reloaded} = 0;
} }
else { else {
my $test_result = run(['nginx', '-t'], timeout => 30); my $caddy = caddy_binary();
if ($test_result->{rc} != 0) { if (!defined $caddy) {
_status_done('caddy not found');
_fail('caddy is not installed: cannot validate the configuration');
$results{caddy_reloaded} = 0;
}
else {
my $validate = run([$caddy, 'validate', '--config', caddy_main_config()],
timeout => 60);
if ($validate->{rc} != 0) {
_status_done('config test failed'); _status_done('config test failed');
my $err = $test_result->{err}; my $err = $validate->{err} . $validate->{out};
$err =~ s/\s+$//; $err =~ s/\s+$//;
_fail("nginx -t failed: $err"); my $tail = length($err) > 500 ? substr($err, -500) : $err;
$results{nginx_reloaded} = 0; _fail("caddy validate failed: $tail");
$results{caddy_reloaded} = 0;
} }
else { else {
my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30); my $reload_result = run(['systemctl', 'reload', 'caddy'], timeout => 30);
if ($reload_result->{rc} == 0) { if ($reload_result->{rc} == 0) {
_status_done('reloaded'); _status_done('reloaded');
$results{nginx_reloaded} = 1; $results{caddy_reloaded} = 1;
} }
else { else {
_status_done('failed'); _status_done('failed');
my $err = $reload_result->{err}; my $err = $reload_result->{err};
$err =~ s/\s+$//; $err =~ s/\s+$//;
_fail("nginx reload failed: $err"); _fail("caddy reload failed: $err");
$results{nginx_reloaded} = 0; $results{caddy_reloaded} = 0;
}
} }
} }
} }
@@ -1885,15 +2216,50 @@ sub uninstall {
$results{env_not_found} = 1; $results{env_not_found} = 1;
} }
# Remove the nginx config. # Remove the caddy drop-in. The main Caddyfile stays: it may carry sites
my $nginx_conf = nginx_conf_path($service_name); # this deployment knows nothing about, and the import line is inert once
_status("Removing nginx $service_name configuration"); # the drop-in is gone.
if (-f $nginx_conf) { my $caddy_conf = caddyfile_path($service_name);
_status("Removing the caddy $service_name drop-in");
if (-f $caddy_conf) {
if ($dry_run) { if ($dry_run) {
_status_done('dry run'); _status_done('dry run');
} }
else { else {
unlink($nginx_conf); unlink($caddy_conf);
if (systemctl_is_active('caddy')) {
my $reload_result = run(['systemctl', 'reload', 'caddy'], timeout => 30);
if ($reload_result->{rc} != 0) {
_status_done('removed (caddy reload failed)');
my $err = $reload_result->{err};
$err =~ s/\s+$//;
_warn("caddy reload failed: $err");
}
else {
_status_done('removed and caddy reloaded');
}
}
else {
_status_done('removed (caddy not running)');
}
$results{caddy_removed} = 1;
}
}
else {
_status_done('not present');
$results{caddy_not_found} = 1;
}
# Remove the drop-in the nginx release wrote, when one is left over.
my $legacy_conf = "$LEGACY_NGINX_DIR/$service_name.conf";
_status('Removing the legacy nginx configuration');
if (-f $legacy_conf) {
if ($dry_run) {
_status_done('dry run');
}
else {
unlink($legacy_conf);
if (systemctl_is_active('nginx')) {
my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30); my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30);
if ($reload_result->{rc} != 0) { if ($reload_result->{rc} != 0) {
_status_done('removed (nginx reload failed)'); _status_done('removed (nginx reload failed)');
@@ -1904,12 +2270,15 @@ sub uninstall {
else { else {
_status_done('removed and nginx reloaded'); _status_done('removed and nginx reloaded');
} }
$results{nginx_removed} = 1; }
else {
_status_done('removed (nginx not running)');
}
$results{legacy_nginx_removed} = 1;
} }
} }
else { else {
_status_done('not present'); _status_done('not present');
$results{nginx_not_found} = 1;
} }
# Remove the TLS certificates. # Remove the TLS certificates.
@@ -1939,7 +2308,9 @@ sub uninstall {
_info('Kept on the system (remove manually if unwanted):'); _info('Kept on the system (remove manually if unwanted):');
_info(" the engine image (podman rmi <image>)"); _info(" the engine image (podman rmi <image>)");
_info(" $state_dir (ModelScope cache with downloaded model weights)"); _info(" $state_dir (ModelScope cache with downloaded model weights)");
_info(" firewalld 'https' rule and the SELinux httpd_can_network_connect boolean"); _info(' the caddy service and /etc/caddy');
_info(" the firewalld 'https' rule (and the SELinux boolean, where a confined "
. 'caddy policy needed it)');
return \%results; return \%results;
} }
@@ -1966,7 +2337,8 @@ sub print_summary {
['unit_removed', 'systemd unit removed'], ['unit_removed', 'systemd unit removed'],
['container_removed', 'engine container removed'], ['container_removed', 'engine container removed'],
['env_removed', 'API key environment file removed'], ['env_removed', 'API key environment file removed'],
['nginx_removed', 'nginx config removed'], ['caddy_removed', 'caddy drop-in removed'],
['legacy_nginx_removed', 'legacy nginx configuration removed'],
['certs_removed', 'TLS certificates removed'], ['certs_removed', 'TLS certificates removed'],
) { ) {
my ($key, $label) = @$pair; my ($key, $label) = @$pair;
@@ -1981,7 +2353,7 @@ sub print_summary {
['unit_not_found', 'systemd unit: already absent'], ['unit_not_found', 'systemd unit: already absent'],
['container_not_found', 'engine container: already absent'], ['container_not_found', 'engine container: already absent'],
['env_not_found', 'API key environment file: already absent'], ['env_not_found', 'API key environment file: already absent'],
['nginx_not_found', 'nginx config: already absent'], ['caddy_not_found', 'caddy drop-in: already absent'],
['certs_not_found', 'TLS certs: already absent'], ['certs_not_found', 'TLS certs: already absent'],
) { ) {
my ($key, $label) = @$pair; my ($key, $label) = @$pair;
@@ -2096,17 +2468,28 @@ sub print_summary {
elsif ($se && $se eq 'absent') { elsif ($se && $se eq 'absent') {
_info('SELinux: not installed (skipped)'); _info('SELinux: not installed (skipped)');
} }
elsif ($se && $se eq 'unconfined') {
_info('SELinux: caddy runs unconfined (nothing to do)');
}
my $ng = $results->{nginx} // {}; my $cd = $results->{caddy} // {};
if ($ng->{nginx_configured} && $ng->{nginx_configured} eq 1 if (defined $cd->{legacy_nginx_removed}) {
&& $ng->{nginx_reloaded} && $ng->{nginx_reloaded} eq 1) { if ($cd->{legacy_nginx_removed} eq 1) {
_ok('nginx: configured and reloaded'); _ok('Legacy nginx configuration: removed');
} }
elsif ($ng->{nginx_configured} && $ng->{nginx_configured} eq 1) { else {
_fail('nginx: config written but reload failed'); _info('Legacy nginx configuration: would be removed');
} }
elsif (($ng->{nginx_configured} // '') eq 'dry run') { }
_info('nginx: would write config and reload'); if ($cd->{caddy_configured} && $cd->{caddy_configured} eq 1
&& $cd->{caddy_reloaded} && $cd->{caddy_reloaded} eq 1) {
_ok('Caddy: configured and reloaded');
}
elsif ($cd->{caddy_configured} && $cd->{caddy_configured} eq 1) {
_fail('Caddy: drop-in written but reload failed');
}
elsif (($cd->{caddy_configured} // '') eq 'dry run') {
_info('Caddy: would write the drop-in and reload');
} }
my $svc = $results->{systemd} // {}; my $svc = $results->{systemd} // {};
@@ -2178,7 +2561,7 @@ Usage: sglang-deploy.pl [options]
--api-key KEY API key for the endpoint (default: generate and --api-key KEY API key for the endpoint (default: generate and
store in /etc/sysconfig) store in /etc/sysconfig)
--dry-run preview without making changes --dry-run preview without making changes
--uninstall tear down the service, container, nginx config --uninstall tear down the service, container, caddy drop-in
and certificates and certificates
--help show this help --help show this help
--version show the version --version show the version
@@ -2340,9 +2723,9 @@ sub is_positive_int {
return defined $value && $value =~ /^\d+$/ && $value + 0 > 0; return defined $value && $value =~ /^\d+$/ && $value + 0 > 0;
} }
# A directory the generated nginx configuration and the unit file carry # A directory the generated Caddyfile drop-in and the unit file carry verbatim:
# verbatim: absolute, and free of the whitespace that splits arguments, of the # absolute, and free of the whitespace that splits arguments and of the %
# % systemd expands as a specifier and of the ; that ends an nginx directive. # systemd expands as a specifier.
sub valid_dir_path { sub valid_dir_path {
my ($path) = @_; my ($path) = @_;
return 0 unless defined $path && length $path; return 0 unless defined $path && length $path;
@@ -2387,7 +2770,7 @@ sub validate_args {
if (!is_positive_int($args->{port}) || $args->{port} + 0 > 65535 if (!is_positive_int($args->{port}) || $args->{port} + 0 > 65535
|| $args->{port} + 0 == 443) { || $args->{port} + 0 == 443) {
_fail("Invalid --port $args->{port}: must be an integer 1-65535 and not 443 " _fail("Invalid --port $args->{port}: must be an integer 1-65535 and not 443 "
. "(nginx)"); . '(Caddy serves 443)');
exit 1; exit 1;
} }
if (!is_number($args->{gpu_memory_utilization}) if (!is_number($args->{gpu_memory_utilization})
@@ -2469,7 +2852,7 @@ sub main {
# ── Deploy path ── # ── Deploy path ──
# 1. System dependencies (before preflight: provides lspci, curl and podman). # 1. System dependencies (before preflight: provides lspci, curl and podman).
print STDERR "\n${BOLD}── System Dependencies ──${RESET}\n"; print STDERR "\n${BOLD}── System Dependencies ──${RESET}\n";
$results{system_deps} = install_system_deps($args->{dry_run}); $results{system_deps} = install_system_deps($os_id, $args->{dry_run});
my @failed_pkgs = @{ $results{system_deps}{failed} // [] }; my @failed_pkgs = @{ $results{system_deps}{failed} // [] };
push @failures, 'failed to install packages: ' . join(', ', @failed_pkgs) if @failed_pkgs; push @failures, 'failed to install packages: ' . join(', ', @failed_pkgs) if @failed_pkgs;
@@ -2501,7 +2884,7 @@ sub main {
exit 1; exit 1;
} }
# 5. TLS certificate (fatal, nginx cannot start without it). # 5. TLS certificate (fatal, caddy cannot start without it).
print STDERR "\n${BOLD}── TLS Certificate ──${RESET}\n"; print STDERR "\n${BOLD}── TLS Certificate ──${RESET}\n";
$results{tls} = setup_tls($args->{cert_dir}, $args->{dry_run}); $results{tls} = setup_tls($args->{cert_dir}, $args->{dry_run});
if (defined $results{tls}{cert_created} && $results{tls}{cert_created} eq 0) { if (defined $results{tls}{cert_created} && $results{tls}{cert_created} eq 0) {
@@ -2524,13 +2907,13 @@ sub main {
push @failures, 'SELinux boolean httpd_can_network_connect not set'; push @failures, 'SELinux boolean httpd_can_network_connect not set';
} }
# 8. nginx. # 8. Caddy.
print STDERR "\n${BOLD}── nginx ──${RESET}\n"; print STDERR "\n${BOLD}── Caddy ──${RESET}\n";
$results{nginx} = setup_nginx( $results{caddy} = setup_caddy(
$args->{port}, $upstream_host, $args->{cert_dir}, $args->{service_name}, $args->{dry_run}, $args->{port}, $upstream_host, $args->{cert_dir}, $args->{service_name}, $args->{dry_run},
); );
if (defined $results{nginx}{nginx_reloaded} && $results{nginx}{nginx_reloaded} eq 0) { if (defined $results{caddy}{caddy_reloaded} && $results{caddy}{caddy_reloaded} eq 0) {
push @failures, 'nginx configuration reload failed'; push @failures, 'caddy configuration reload failed';
} }
# 9. systemd service (the model is probed before the unit is written). # 9. systemd service (the model is probed before the unit is written).
+188 -24
View File
@@ -16,7 +16,11 @@ my $LOG = "$WORK/log/stubs.log";
my $SCRIPT = $ENV{SGLANG_RIG_SCRIPT} // "$RIG/../../sglang-deploy.pl"; my $SCRIPT = $ENV{SGLANG_RIG_SCRIPT} // "$RIG/../../sglang-deploy.pl";
my $UNIT = '/etc/systemd/system/sglang.service'; my $UNIT = '/etc/systemd/system/sglang.service';
my $ENVFILE = '/etc/sysconfig/sglang'; my $ENVFILE = '/etc/sysconfig/sglang';
my $NGINX = '/etc/nginx/conf.d/sglang.conf'; my $CADDY = '/etc/caddy/Caddyfile.d/sglang.caddyfile';
my $CADDY_MAIN = '/etc/caddy/Caddyfile';
my $CADDY_UNIT = '/etc/systemd/system/caddy.service';
my $CADDY_BIN = '/usr/local/bin/caddy';
my $NGINX_LEGACY = '/etc/nginx/conf.d/sglang.conf';
my $CERTDIR = '/etc/ssl/sglang'; my $CERTDIR = '/etc/ssl/sglang';
my $STATEDIR = '/opt/sglang'; my $STATEDIR = '/opt/sglang';
@@ -105,22 +109,27 @@ sub mode_of {
} }
sub reset_fixture { sub reset_fixture {
for my $path ($UNIT, $ENVFILE, $NGINX) { for my $path ($UNIT, $ENVFILE, $CADDY, $CADDY_MAIN, $CADDY_UNIT, $CADDY_BIN,
$NGINX_LEGACY) {
unlink($path); unlink($path);
} }
remove_tree('/etc/caddy');
remove_tree($CERTDIR); remove_tree($CERTDIR);
remove_tree($STATEDIR); remove_tree($STATEDIR);
remove_tree($FIX); remove_tree($FIX);
remove_tree($ST); remove_tree($ST);
# Directories a host that ran server-setup.pl has: nginx's configuration drop-in # Directories a host that ran server-setup.pl has: the legacy nginx drop-in
# and systemd's unit directory. # directory an earlier release wrote into, and systemd's unit directory.
make_dirs($FIX, $ST, "$WORK/log", '/etc/nginx/conf.d', '/etc/systemd/system'); make_dirs($FIX, $ST, "$WORK/log", '/etc/nginx/conf.d', '/etc/systemd/system');
my $fh; my $fh;
open($fh, q{>}, $LOG) and close($fh); open($fh, q{>}, $LOG) and close($fh);
# Packages a real host or a previous run has already installed. # Packages a real host or a previous run has already installed. nginx stands
# for the drop-in the previous release left behind.
write_fixture('rpm-installed', "nginx\n"); write_fixture('rpm-installed', "nginx\n");
write_fixture('fw-services', "\n"); write_fixture('fw-services', "\n");
# firewalld is running: server-setup.pl ensures it, and the firewall step needs it. # firewalld is running: server-setup.pl ensures it, and the firewall step needs it.
# The handle is declared first: a my inside the open's argument list does not
# reach the right-hand operand of the and on this interpreter.
my $fw; my $fw;
open($fw, '>', "$ST/active.firewalld") and close($fw); open($fw, '>', "$ST/active.firewalld") and close($fw);
return; return;
@@ -184,8 +193,8 @@ sub reset_log {
# so the script's own PATH lookup finds them and nothing of the real system is used. # so the script's own PATH lookup finds them and nothing of the real system is used.
sub prepare_stubs { sub prepare_stubs {
make_dirs($BIN, $FIX, $ST, "$WORK/log"); make_dirs($BIN, $FIX, $ST, "$WORK/log");
for my $name (qw(lspci rpm dnf podman systemctl curl openssl nginx for my $name (qw(lspci rpm dnf podman systemctl curl openssl caddy tar useradd
firewall-cmd getenforce getsebool setsebool)) { semodule firewall-cmd getenforce getsebool setsebool)) {
my $link = "$BIN/$name"; my $link = "$BIN/$name";
# A link left over from a work directory that moved reads as broken to # A link left over from a work directory that moved reads as broken to
# -e, and its stale target would leave the stubs unreachable: it is # -e, and its stale target would leave the stubs unreachable: it is
@@ -197,6 +206,21 @@ sub prepare_stubs {
return; return;
} }
# The caddy package creates its group; the rig creates it the same way, so the
# key permission the package makes possible is exercised for real. The group
# file is edited directly: the minimal openEuler image carries no groupadd to
# call, and a group entry is all the getgrnam in the script needs.
sub prepare_group {
return if defined getgrnam('caddy');
my $existing = slurp_file('/etc/group');
my $gid = 995;
$gid++ while $existing =~ /^[^:]+:[^:]*:\Q$gid\E:/m;
open(my $fh, '>>', '/etc/group') or die "cannot append to /etc/group: $!\n";
print {$fh} "caddy:x:$gid:\n";
close($fh);
return;
}
sub prepare_devices { sub prepare_devices {
# The driver creates these on a real host; the rig fakes them (needs --privileged). # The driver creates these on a real host; the rig fakes them (needs --privileged).
return if -e q{/dev/kfd}; return if -e q{/dev/kfd};
@@ -214,12 +238,15 @@ sub scenario_fresh {
check($rc == 0, 'fresh: exit 0'); check($rc == 0, 'fresh: exit 0');
check(-f $UNIT, 'fresh: unit written'); check(-f $UNIT, 'fresh: unit written');
check(-f $ENVFILE, 'fresh: environment file written'); check(-f $ENVFILE, 'fresh: environment file written');
check(-f $NGINX, 'fresh: nginx configuration written'); check(-f $CADDY, 'fresh: caddy drop-in written');
check(-f $CADDY_MAIN, 'fresh: main Caddyfile written');
check(-f "$CERTDIR/sglang.crt" && -f "$CERTDIR/sglang.key", 'fresh: certificate written'); check(-f "$CERTDIR/sglang.crt" && -f "$CERTDIR/sglang.key", 'fresh: certificate written');
check(-d "$STATEDIR/modelscope", 'fresh: model cache directory created'); check(-d "$STATEDIR/modelscope", 'fresh: model cache directory created');
check(mode_of($ENVFILE) eq '0600', 'fresh: environment file is 0600 (' . mode_of($ENVFILE) . ')'); check(mode_of($ENVFILE) eq '0600', 'fresh: environment file is 0600 (' . mode_of($ENVFILE) . ')');
check(mode_of("$CERTDIR/sglang.key") eq '0600', 'fresh: key is 0600'); check(mode_of("$CERTDIR/sglang.key") eq '0640', 'fresh: key is 0640 for the caddy group (' . mode_of("$CERTDIR/sglang.key") . ')');
check(mode_of("$CERTDIR/sglang.crt") eq '0644', 'fresh: certificate is 0644'); check(mode_of("$CERTDIR/sglang.crt") eq '0644', 'fresh: certificate is 0644');
check((stat("$CERTDIR/sglang.key"))[5] == getgrnam('caddy'),
'fresh: the key belongs to the caddy group');
my $env = slurp_file($ENVFILE); my $env = slurp_file($ENVFILE);
check_like($env, qr/^SGLANG_API_KEY=([A-Za-z0-9_-]{43})\n$/, 'fresh: generated key, url-safe, 43 chars'); check_like($env, qr/^SGLANG_API_KEY=([A-Za-z0-9_-]{43})\n$/, 'fresh: generated key, url-safe, 43 chars');
@@ -231,19 +258,29 @@ sub scenario_fresh {
check_like($unit, qr/--mem-fraction-static 0\.9/, 'fresh: memory fraction default'); check_like($unit, qr/--mem-fraction-static 0\.9/, 'fresh: memory fraction default');
check_unlike($unit, qr/SGLANG_USE_AITER/, 'fresh: no Radeon variables on an Instinct host'); check_unlike($unit, qr/SGLANG_USE_AITER/, 'fresh: no Radeon variables on an Instinct host');
my $nginx = slurp_file($NGINX); my $caddy = slurp_file($CADDY);
check_like($nginx, qr|proxy_pass http://\[::1\]:8000;|, 'fresh: nginx proxies to the loopback engine'); check_like($caddy, qr/reverse_proxy \[::1\]:8000 \{/, 'fresh: caddy proxies to the loopback engine');
check_like($nginx, qr|ssl_certificate /etc/ssl/sglang/sglang\.crt;|, 'fresh: nginx uses the sglang certificate'); check_like($caddy, qr|tls /etc/ssl/sglang/sglang\.crt /etc/ssl/sglang/sglang\.key|,
'fresh: caddy uses the sglang certificate pair');
check_like($caddy, qr/flush_interval -1/, 'fresh: streaming is unbuffered');
my $main = slurp_file($CADDY_MAIN);
check_like($main, qr/^import Caddyfile\.d\/\*\.caddyfile$/m, 'fresh: the main Caddyfile imports the drop-ins');
check(count_in_log(qr/^podman pull /) == 1, 'fresh: exactly one image pull'); check(count_in_log(qr/^podman pull /) == 1, 'fresh: exactly one image pull');
check_like(stub_log(), qr/^podman pull docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x$/m, check_like(stub_log(), qr/^podman pull docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x$/m,
'fresh: the pulled image is the resolved tag'); 'fresh: the pulled image is the resolved tag');
check_like(stub_log(), qr/^caddy version$/m, 'fresh: caddy is reported from the binary');
check_like(stub_log(), qr/^caddy validate --config \/etc\/caddy\/Caddyfile$/m,
'fresh: the configuration is validated before the reload');
check_like(stub_log(), qr/^systemctl enable --now caddy$/m, 'fresh: caddy enabled and started');
check_like(stub_log(), qr/^systemctl reload caddy$/m, 'fresh: caddy reloaded');
check(count_in_log(qr/^systemctl enable sglang$/) == 1, 'fresh: service enabled'); check(count_in_log(qr/^systemctl enable sglang$/) == 1, 'fresh: service enabled');
check(count_in_log(qr/^systemctl start sglang$/) == 1, 'fresh: service started'); check(count_in_log(qr/^systemctl start sglang$/) == 1, 'fresh: service started');
check_like(stub_log(), qr/^firewall-cmd --permanent --add-service=https$/m, 'fresh: HTTPS opened'); check_like(stub_log(), qr/^firewall-cmd --permanent --add-service=https$/m, 'fresh: HTTPS opened');
check_like($out, qr/Engine image: docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x/, check_like($out, qr/Engine image: docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x/,
'fresh: summary names the image'); 'fresh: summary names the image');
check_like($out, qr/systemd: sglang running on \[::1\]:8000/, 'fresh: summary names the endpoint'); check_like($out, qr/systemd: sglang running on \[::1\]:8000/, 'fresh: summary names the endpoint');
check_like($out, qr/Caddy: configured and reloaded/, 'fresh: summary reports caddy');
check_like($out, qr/API key \(shown once, store it securely\)/, 'fresh: the generated key is shown once'); check_like($out, qr/API key \(shown once, store it securely\)/, 'fresh: the generated key is shown once');
check_unlike($out, qr/✗/, 'fresh: no failed step'); check_unlike($out, qr/✗/, 'fresh: no failed step');
return; return;
@@ -261,6 +298,7 @@ sub scenario_rerun {
check_like(stub_log(), qr/^podman image exists /m, 'rerun: the image is checked instead'); check_like(stub_log(), qr/^podman image exists /m, 'rerun: the image is checked instead');
check(count_in_log(qr/^systemctl enable sglang$/) == 0, 'rerun: no second enable'); check(count_in_log(qr/^systemctl enable sglang$/) == 0, 'rerun: no second enable');
check(count_in_log(qr/^systemctl start sglang$/) == 0, 'rerun: no second start'); check(count_in_log(qr/^systemctl start sglang$/) == 0, 'rerun: no second start');
check(count_in_log(qr/^systemctl enable --now caddy$/) == 0, 'rerun: no second caddy enable');
check(count_in_log(qr/try-restart/) == 0, 'rerun: no restart, the unit did not change'); check(count_in_log(qr/try-restart/) == 0, 'rerun: no restart, the unit did not change');
check(count_in_log(qr/^dnf install /) == 0, 'rerun: no second package transaction'); check(count_in_log(qr/^dnf install /) == 0, 'rerun: no second package transaction');
check(slurp_file($ENVFILE) eq $key_before, 'rerun: the API key is reused, not regenerated'); check(slurp_file($ENVFILE) eq $key_before, 'rerun: the API key is reused, not regenerated');
@@ -282,7 +320,8 @@ sub scenario_dry_run {
check($rc == 0, 'dry run: exit 0'); check($rc == 0, 'dry run: exit 0');
check(!-f $UNIT, 'dry run: no unit written'); check(!-f $UNIT, 'dry run: no unit written');
check(!-f $ENVFILE, 'dry run: no environment file written'); check(!-f $ENVFILE, 'dry run: no environment file written');
check(!-f $NGINX, 'dry run: no nginx configuration written'); check(!-f $CADDY, 'dry run: no caddy drop-in written');
check(!-e '/etc/caddy', 'dry run: no caddy directory created');
check(!-d $CERTDIR, 'dry run: no certificate directory'); check(!-d $CERTDIR, 'dry run: no certificate directory');
check(count_in_log(qr/^podman pull /) == 0, 'dry run: no pull'); check(count_in_log(qr/^podman pull /) == 0, 'dry run: no pull');
check(count_in_log(qr/^systemctl (enable|start) /) == 0, 'dry run: no service change'); check(count_in_log(qr/^systemctl (enable|start) /) == 0, 'dry run: no service change');
@@ -302,14 +341,17 @@ sub scenario_uninstall {
check($rc == 0, 'uninstall: exit 0'); check($rc == 0, 'uninstall: exit 0');
check(!-f $UNIT, 'uninstall: unit removed'); check(!-f $UNIT, 'uninstall: unit removed');
check(!-f $ENVFILE, 'uninstall: environment file removed'); check(!-f $ENVFILE, 'uninstall: environment file removed');
check(!-f $NGINX, 'uninstall: nginx configuration removed'); check(!-f $CADDY, 'uninstall: caddy drop-in removed');
check(-f $CADDY_MAIN, 'uninstall: the main Caddyfile is kept');
check(!-d $CERTDIR, 'uninstall: certificate directory removed'); check(!-d $CERTDIR, 'uninstall: certificate directory removed');
check(-d $STATEDIR, 'uninstall: model cache kept'); check(-d $STATEDIR, 'uninstall: model cache kept');
check(-e "$ST/image.docker.io_lmsysorg_sglang_v0.5.19-rocm724-mi30x", check(-e "$ST/image.docker.io_lmsysorg_sglang_v0.5.19-rocm724-mi30x",
'uninstall: the image is kept in podman'); 'uninstall: the image is kept in podman');
check_like(stub_log(), qr/^systemctl stop sglang$/m, 'uninstall: service stopped'); check_like(stub_log(), qr/^systemctl stop sglang$/m, 'uninstall: service stopped');
check_like(stub_log(), qr/^systemctl disable sglang$/m, 'uninstall: service disabled'); check_like(stub_log(), qr/^systemctl disable sglang$/m, 'uninstall: service disabled');
check_like(stub_log(), qr/^systemctl reload caddy$/m, 'uninstall: caddy reloaded after the drop-in');
check_like($out, qr/SGLang service stopped/, 'uninstall: summary reports the stop'); check_like($out, qr/SGLang service stopped/, 'uninstall: summary reports the stop');
check_like($out, qr/caddy drop-in removed/, 'uninstall: summary reports the drop-in');
check_like($out, qr/Kept on the system/, 'uninstall: the kept state is listed'); check_like($out, qr/Kept on the system/, 'uninstall: the kept state is listed');
check_like($out, qr/ModelScope cache/, 'uninstall: the cache is named as kept'); check_like($out, qr/ModelScope cache/, 'uninstall: the cache is named as kept');
return; return;
@@ -322,6 +364,78 @@ sub scenario_uninstall_twice {
check($rc == 0, 'uninstall twice: exit 0'); check($rc == 0, 'uninstall twice: exit 0');
check_like($out, qr/already absent/, 'uninstall twice: idempotent'); check_like($out, qr/already absent/, 'uninstall twice: idempotent');
check(count_in_log(qr/^systemctl stop /) == 0, 'uninstall twice: nothing to stop'); check(count_in_log(qr/^systemctl stop /) == 0, 'uninstall twice: nothing to stop');
check(count_in_log(qr/^systemctl reload caddy$/) == 0,
'uninstall twice: no reload without a drop-in');
return;
}
# A host deployed by the nginx release carries its drop-in. Both a deploy and
# an uninstall remove it, so exactly one proxy owns :443 afterwards.
sub scenario_legacy_nginx {
reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n");
open(my $fh, '>', $NGINX_LEGACY) or die "cannot write $NGINX_LEGACY: $!\n";
print {$fh} "server {\n listen 443 ssl;\n}\n";
close($fh);
# The stub state: nginx is running on this host, so the removal reloads it.
my $st;
open($st, '>', "$ST/active.nginx") and close($st);
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
check($rc == 0, 'legacy nginx: exit 0');
check(!-f $NGINX_LEGACY, 'legacy nginx: the old drop-in is gone on deploy');
check_like(stub_log(), qr/^systemctl reload nginx$/m,
'legacy nginx: the running nginx is reloaded');
check_like($out, qr/Legacy nginx configuration: removed/, 'legacy nginx: the summary names it');
# An uninstall on a host the new release never deployed cleans it too.
reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n");
open($fh, '>', $NGINX_LEGACY) or die "cannot write $NGINX_LEGACY: $!\n";
print {$fh} "server {\n listen 443 ssl;\n}\n";
close($fh);
reset_log();
($rc, $out) = run_script('--uninstall');
check($rc == 0, 'legacy nginx: uninstall exit 0');
check(!-f $NGINX_LEGACY, 'legacy nginx: the old drop-in is gone on uninstall');
check_like($out, qr/legacy nginx configuration removed/, 'legacy nginx: the uninstall summary names it');
return;
}
# Where no repository carries the caddy package, the official release binary
# takes its place: download, extract, install, the service user, and the unit
# file the package would have carried.
sub scenario_caddy_binary {
reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n");
write_fixture('caddy-no-package', "1\n");
unlink('/usr/bin/caddy'); # order independence: no package binary, no stub
unlink("$BIN/caddy") or die "cannot remove the caddy stub: $!\n";
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
check($rc == 0, 'caddy binary: exit 0');
check_like(stub_log(), qr/^dnf install -y caddy$/m, 'caddy binary: the package install was attempted');
check_like(stub_log(), qr{^curl -fsSL -o \S+ https://github\.com/caddyserver/caddy/releases/download/v2\.10\.2/caddy_2\.10\.2_linux_amd64\.tar\.gz$}m,
'caddy binary: the release asset is downloaded');
check_like(stub_log(), qr/^tar -xzf \S+ -C \S+$/m, 'caddy binary: the archive is extracted');
check(-x $CADDY_BIN, 'caddy binary: the binary is installed executable');
check_like(stub_log(), qr/^useradd --system --home-dir \/var\/lib\/caddy --create-home --shell \/sbin\/nologin caddy$/m,
'caddy binary: the service user is created');
check(-f $CADDY_UNIT, 'caddy binary: the unit file is written');
my $unit = slurp_file($CADDY_UNIT);
check_like($unit, qr|ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile|,
'caddy binary: the unit runs the release binary');
check_like(stub_log(), qr/^systemctl daemon-reload$/m, 'caddy binary: systemd reloaded');
check_like(stub_log(), qr{^caddy validate --config /etc/caddy/Caddyfile$}m,
'caddy binary: the installed binary validates');
check_like($out, qr/Caddy: configured and reloaded/, 'caddy binary: the deployment completes');
unlink($CADDY_BIN);
unlink($CADDY_UNIT);
unlink("$FIX/caddy-no-package");
symlink("$RIG/stub.pl", "$BIN/caddy") or die "cannot restore the caddy stub: $!\n";
return; return;
} }
@@ -495,7 +609,7 @@ sub scenario_validation {
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n"); write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--port', '443', '--dry-run'); my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--port', '443', '--dry-run');
check($rc == 1, 'validation: port 443 refused'); check($rc == 1, 'validation: port 443 refused');
check_like($out, qr/must be 1-65535 and not 443/, 'validation: port message'); check_like($out, qr/must be an integer 1-65535 and not 443/, 'validation: port message');
($rc, $out) = run_script('--model', 'not-a-model-id', '--dry-run'); ($rc, $out) = run_script('--model', 'not-a-model-id', '--dry-run');
check($rc == 1, 'validation: bad model ID refused'); check($rc == 1, 'validation: bad model ID refused');
check_like($out, qr/Invalid model ID/, 'validation: model message'); check_like($out, qr/Invalid model ID/, 'validation: model message');
@@ -545,7 +659,7 @@ sub scenario_non_root {
my $rc = $? >> 8; my $rc = $? >> 8;
my $out = slurp_file($out_file); my $out = slurp_file($out_file);
check($rc == 0, 'non-root: --version works without root'); check($rc == 0, 'non-root: --version works without root');
check_like($out, qr/^sglang-deploy\.pl 2\.0\.0$/, 'non-root: the version is printed'); check_like($out, qr/^sglang-deploy\.pl 2\.1\.0$/, 'non-root: the version is printed');
my $pid3 = fork(); my $pid3 = fork();
die "cannot fork: $!\n" unless defined $pid3; die "cannot fork: $!\n" unless defined $pid3;
@@ -583,12 +697,12 @@ sub scenario_dependency_section {
reset_fixture(); reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n"); write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n"); write_fixture('rocm.txt', "7.2.4\n");
write_fixture('rpm-installed', "pciutils\ncurl\nopenssl\npodman\nnginx\n"); write_fixture('rpm-installed', "pciutils\ncurl\nopenssl\npodman\ntar\n");
reset_log(); reset_log();
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--dry-run'); my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--dry-run');
check($rc == 0, 'deps: exit 0'); check($rc == 0, 'deps: exit 0');
check(count_in_log(qr/^dnf install /) == 0, 'deps: no transaction when all packages are present'); check(count_in_log(qr/^dnf install /) == 0, 'deps: no transaction when all packages are present');
check_like($out, qr/All 4 packages already installed/, 'deps: reported as present'); check_like($out, qr/All 5 packages already installed/, 'deps: reported as present');
reset_fixture(); reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n"); write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
@@ -596,10 +710,41 @@ sub scenario_dependency_section {
write_fixture('rpm-installed', "\n"); write_fixture('rpm-installed', "\n");
reset_log(); reset_log();
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3'); ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
check_like(stub_log(), qr/^dnf install -y pciutils curl openssl podman$/m, check_like(stub_log(), qr/^dnf install -y pciutils curl openssl podman tar$/m,
'deps: the four packages are installed in one transaction'); 'deps: the five packages are installed in one transaction');
check_like($out, qr/nginx \(reverse proxy\) is not installed: run server-setup.pl first/, check_like(stub_log(), qr/^caddy version$/m, 'deps: caddy is checked after the transaction');
'deps: the missing reverse proxy is warned about'); check_unlike($out, qr/run server-setup\.pl first/,
'deps: no warning points at server-setup.pl, caddy is installed here');
return;
}
# CentOS Stream carries caddy in EPEL, and the repository file installs first,
# which is what makes the package transaction below it resolvable.
sub scenario_epel {
reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n");
my $real = slurp_file('/etc/os-release');
open(my $fh, '>', '/etc/os-release') or die "cannot write /etc/os-release: $!\n";
print {$fh} "ID=centos\nVERSION_ID=\"10\"\n";
close($fh);
unlink('/usr/bin/caddy'); # order independence: no binary, no stub
unlink("$BIN/caddy");
reset_log();
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
open(my $back, '>', '/etc/os-release') or die "cannot restore /etc/os-release: $!\n";
print {$back} $real;
close($back);
check($rc == 0, 'epel: exit 0');
check_like(stub_log(), qr/^dnf install -y epel-release$/m, 'epel: the repository file installs first');
check_like(stub_log(), qr/^dnf install -y caddy$/m, 'epel: the package installs after it');
check_like($out, qr/Installing caddy\.\.\. package/, 'epel: the package path is taken');
check_like($out, qr/Caddy: configured and reloaded/, 'epel: the deployment completes');
unlink('/usr/bin/caddy');
unlink("$FIX/caddy-no-package");
symlink("$RIG/stub.pl", "$BIN/caddy") or die "cannot restore the caddy stub: $!\n";
return; return;
} }
@@ -619,7 +764,7 @@ sub scenario_custom_layout {
); );
check($rc == 0, 'custom layout: exit 0'); check($rc == 0, 'custom layout: exit 0');
check(-f '/etc/systemd/system/llm.service', 'custom layout: the named unit is written'); check(-f '/etc/systemd/system/llm.service', 'custom layout: the named unit is written');
check(-f '/etc/nginx/conf.d/llm.conf', 'custom layout: the named nginx file is written'); check(-f '/etc/caddy/Caddyfile.d/llm.caddyfile', 'custom layout: the named caddy drop-in is written');
# The certificate file names follow the program, as they did before, not the # The certificate file names follow the program, as they did before, not the
# service name; the directory follows --cert-dir. # service name; the directory follows --cert-dir.
check(-f '/etc/ssl/llm/sglang.crt', 'custom layout: certificates follow the directory'); check(-f '/etc/ssl/llm/sglang.crt', 'custom layout: certificates follow the directory');
@@ -632,7 +777,7 @@ sub scenario_custom_layout {
check_like($unit, qr|--volume /srv/llm/modelscope:/root/\.cache/modelscope:Z|, check_like($unit, qr|--volume /srv/llm/modelscope:/root/\.cache/modelscope:Z|,
'custom layout: the cache volume follows the state directory'); 'custom layout: the cache volume follows the state directory');
unlink('/etc/systemd/system/llm.service'); unlink('/etc/systemd/system/llm.service');
unlink('/etc/nginx/conf.d/llm.conf'); unlink('/etc/caddy/Caddyfile.d/llm.caddyfile');
remove_tree('/etc/ssl/llm'); remove_tree('/etc/ssl/llm');
remove_tree('/srv/llm'); remove_tree('/srv/llm');
return; return;
@@ -674,17 +819,32 @@ sub scenario_selinux {
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set while permissive'); check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set while permissive');
check_like($out, qr/SELinux: permissive \(skipped\)/, 'selinux: reported as skipped'); check_like($out, qr/SELinux: permissive \(skipped\)/, 'selinux: reported as skipped');
# Enforcing with no confined caddy policy: the distributions run caddy
# unconfined, so no boolean is touched.
reset_fixture(); reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n"); write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n"); write_fixture('rocm.txt', "7.2.4\n");
$ENV{STUB_SELINUX} = 'Enforcing'; $ENV{STUB_SELINUX} = 'Enforcing';
reset_log(); reset_log();
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3'); ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
check(count_in_log(qr/^setsebool /) == 0, 'selinux: no boolean without a confined policy');
check_like($out, qr/SELinux: caddy runs unconfined \(nothing to do\)/,
'selinux: the unconfined case is stated');
# Enforcing with a confined caddy policy loaded: the boolean is set.
reset_fixture();
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
write_fixture('rocm.txt', "7.2.4\n");
write_fixture('semodule-caddy', "1\n");
reset_log();
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
check_like(stub_log(), qr/^semodule -l$/m, 'selinux: the loaded modules are asked for');
check_like(stub_log(), qr/^setsebool -P httpd_can_network_connect=1$/m, 'selinux: the boolean is set'); check_like(stub_log(), qr/^setsebool -P httpd_can_network_connect=1$/m, 'selinux: the boolean is set');
check_like($out, qr/SELinux: httpd_can_network_connect on/, 'selinux: reported as on'); check_like($out, qr/SELinux: httpd_can_network_connect on/, 'selinux: reported as on');
reset_log(); reset_log();
my ($rc2, $out2) = run_script('--model', 'ZhipuAI/GLM-5.3'); my ($rc2, $out2) = run_script('--model', 'ZhipuAI/GLM-5.3');
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set when already on'); check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set when already on');
unlink("$FIX/semodule-caddy");
delete $ENV{STUB_SELINUX}; delete $ENV{STUB_SELINUX};
return; return;
} }
@@ -695,6 +855,9 @@ my %scenarios = (
dry_run => \&scenario_dry_run, dry_run => \&scenario_dry_run,
uninstall => \&scenario_uninstall, uninstall => \&scenario_uninstall,
uninstall_twice => \&scenario_uninstall_twice, uninstall_twice => \&scenario_uninstall_twice,
legacy_nginx => \&scenario_legacy_nginx,
caddy_binary => \&scenario_caddy_binary,
epel => \&scenario_epel,
radeon => \&scenario_radeon, radeon => \&scenario_radeon,
radeon_dev => \&scenario_radeon_dev, radeon_dev => \&scenario_radeon_dev,
radeon_with_image => \&scenario_radeon_with_image, radeon_with_image => \&scenario_radeon_with_image,
@@ -715,6 +878,7 @@ my %scenarios = (
); );
prepare_stubs(); prepare_stubs();
prepare_group();
prepare_devices(); prepare_devices();
my @wanted = @ARGV ? @ARGV : sort keys %scenarios; my @wanted = @ARGV ? @ARGV : sort keys %scenarios;
Regular → Executable
+71 -5
View File
@@ -74,6 +74,12 @@ if ($name eq 'rpm') {
if ($name eq 'dnf') { if ($name eq 'dnf') {
my @pkgs = grep { !/^-/ && $_ ne 'install' } @args; my @pkgs = grep { !/^-/ && $_ ne 'install' } @args;
# Parenthesised on purpose: a named list operator swallows a trailing &&,
# and the unparenthesised form asks the grep about a boolean, not the list.
if ((grep { $_ eq 'caddy' } @pkgs) && -f "$FIX/caddy-no-package") {
print STDERR "Error: Unable to find a match: caddy\n";
exit 1;
}
my $installed = fixture_text('rpm-installed', ''); my $installed = fixture_text('rpm-installed', '');
for my $pkg (@pkgs) { for my $pkg (@pkgs) {
$installed .= "$pkg\n" unless $installed =~ /^\Q$pkg\E$/m; $installed .= "$pkg\n" unless $installed =~ /^\Q$pkg\E$/m;
@@ -82,6 +88,12 @@ if ($name eq 'dnf') {
print {$fh} $installed; print {$fh} $installed;
close($fh); close($fh);
} }
# A package transaction that installs caddy leaves the binary where PATH
# finds it, the way the real package does. The grep is parenthesised as
# above: a named list operator swallows a trailing &&.
if ((grep { $_ eq 'caddy' } @pkgs) && !-e '/usr/bin/caddy') {
symlink($0, '/usr/bin/caddy');
}
print "Installing: @pkgs\n"; print "Installing: @pkgs\n";
exit 0; exit 0;
} }
@@ -138,14 +150,32 @@ if ($name eq 'curl') {
print "\n__HTTP__$code\n" if $joined =~ /__HTTP__/; print "\n__HTTP__$code\n" if $joined =~ /__HTTP__/;
exit 0; exit 0;
} }
if ($url =~ m{api\.github\.com/repos/caddyserver/caddy}) {
print fixture_text('caddy-release.json', qq({"tag_name":"v2.10.2"}\n));
exit 0;
}
if ($url =~ m{api\.github\.com}) { if ($url =~ m{api\.github\.com}) {
print fixture_text('releases.json', qq({"tag_name":"v0.5.19"}\n)); print fixture_text('releases.json', qq({"tag_name":"v0.5.19"}\n));
exit 0; exit 0;
} }
if ($url =~ m{github\.com/caddyserver/caddy/releases/download/}) {
my $dest;
for my $i (0 .. $#args) {
$dest = $args[$i + 1] if $args[$i] eq '-o';
}
if (defined $dest) {
open(my $fh, '>', $dest) or exit 1;
print {$fh} "stub caddy release archive\n";
close($fh);
}
exit 0;
}
if ($url =~ m{hub\.docker\.com}) { if ($url =~ m{hub\.docker\.com}) {
if (-f "$FIX/image-missing") { # A definitive 404 on a tag lookup is what the script reads as
print STDERR "curl: (22) The requested URL returned error: 404\n"; # unpublished; a curl-level failure would read as cannot-tell.
exit 22; if ($url =~ m{/tags/[A-Za-z0-9._-]+$} && -f "$FIX/image-missing") {
print "404";
exit 0;
} }
if ($url =~ /tags\?/) { if ($url =~ /tags\?/) {
# A tag listing, newest first: the rig's AMD development build. # A tag listing, newest first: the rig's AMD development build.
@@ -181,8 +211,44 @@ if ($name eq 'openssl') {
exit 0; exit 0;
} }
if ($name eq 'nginx') { if ($name eq 'caddy') {
print "nginx: configuration file /etc/nginx/nginx.conf test is successful\n"; my $joined = join(' ', @args);
if ($joined =~ /version/) {
print "v2.10.2 h1:stub\n";
exit 0;
}
if ($joined =~ /validate/) {
print "Valid configuration\n";
exit 0;
}
exit 0;
}
if ($name eq 'tar') {
# The release-binary install extracts the archive and installs the binary it
# names; the stub lays down a link to this dispatcher, so the installed
# stand-in answers and logs like every other stubbed command.
my $dest_dir;
for my $i (0 .. $#args) {
$dest_dir = $args[$i + 1] if $args[$i] eq '-C';
}
if (defined $dest_dir) {
unlink("$dest_dir/caddy");
symlink($0, "$dest_dir/caddy");
}
exit 0;
}
if ($name eq 'useradd') {
exit 0;
}
if ($name eq 'semodule') {
# A loaded module line looks like "100 caddy\tpp"; the fixture decides
# whether this host carries a confined caddy policy.
if (-f "$FIX/semodule-caddy") {
print "100 caddy\tpp\n";
}
exit 0; exit 0;
} }
+19
View File
@@ -124,6 +124,25 @@ my %defaults = parse_args();
is($defaults{dry_run}, 0, 'args: dry run is off by default'); is($defaults{dry_run}, 0, 'args: dry run is off by default');
@ARGV = @saved; @ARGV = @saved;
# ---------------------------------------------------------------------------
# The base package catalogue
# ---------------------------------------------------------------------------
# The proxy of the collection is caddy everywhere; nginx left the baseline when
# sglang-deploy.pl moved to Caddy, and the name survives only in its own legacy
# clean-up.
check((grep { $_ eq 'caddy' } base_packages()) == 1, 'packages: caddy is a base package');
check((grep { $_ eq 'nginx' } base_packages()) == 0, 'packages: nginx is not');
is(join('|', base_packages()),
'nano|curl|wget|htop|tmux|rsync|caddy|openssl|jq|fastfetch',
'packages: the whole list, in order');
my %openeuler_gaps = unpackaged_for('openeuler');
is(exists $openeuler_gaps{caddy} ? 'yes' : 'no', 'yes',
'packages: the openEuler gap is named in the unpackaged map');
is(scalar(unpackaged_for('fedora')) // 0, 0, 'packages: Fedora has no unpackaged entry');
is(scalar(unpackaged_for('centos')) // 0, 0,
'packages: CentOS Stream has no unpackaged entry, EPEL carries caddy');
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# The command runner # The command runner
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+37 -16
View File
@@ -138,7 +138,7 @@ is((valid_dir_path('/opt/sg lang') ? 1 : 0), 0, 'valid_dir_path: whitespace is r
is((valid_dir_path('/opt/%h/sglang') ? 1 : 0), 0, is((valid_dir_path('/opt/%h/sglang') ? 1 : 0), 0,
'valid_dir_path: a systemd specifier is refused'); 'valid_dir_path: a systemd specifier is refused');
is((valid_dir_path('/opt/x;/sglang') ? 1 : 0), 0, is((valid_dir_path('/opt/x;/sglang') ? 1 : 0), 0,
'valid_dir_path: an nginx directive end is refused'); 'valid_dir_path: a path with a semicolon is refused');
# ---- run(): exit status, signals and timeout ------------------------------ # ---- run(): exit status, signals and timeout ------------------------------
my $killed = run([$^X, '-e', 'kill 9, $$']); my $killed = run([$^X, '-e', 'kill 9, $$']);
@@ -164,22 +164,43 @@ is(scalar @{ radeon_env_for(undef, 1) }, 2,
'env: a custom image on a Radeon-only host carries the Radeon defaults'); 'env: a custom image on a Radeon-only host carries the Radeon defaults');
is(scalar @{ radeon_env_for('mi30x', 0) }, 0, 'env: an Instinct host carries none'); is(scalar @{ radeon_env_for('mi30x', 0) }, 0, 'env: an Instinct host carries none');
# ---- nginx config -------------------------------------------------------- # ---- caddy drop-in --------------------------------------------------------
my $conf = nginx_conf_content(8000, '[::1]', '/etc/ssl/sglang'); my $conf = caddyfile_content(8000, '[::1]', '/etc/ssl/sglang');
like($conf, qr/listen \[::\]:443 ssl;/, 'nginx: IPv6 listener on a dual-stack kernel'); like($conf, qr/^:443 \{$/m, 'caddy: the endpoint site on 443');
like($conf, qr/listen 443 ssl;/, 'nginx: IPv4 listener'); like($conf, qr|tls /etc/ssl/sglang/sglang\.crt /etc/ssl/sglang/sglang\.key|,
like($conf, qr|ssl_certificate /etc/ssl/sglang/sglang\.crt;|, 'nginx: certificate path'); 'caddy: certificate pair paths');
like($conf, qr/ssl_certificate_key \/etc\/ssl\/sglang\/sglang\.key;/, 'nginx: key path'); like($conf, qr/reverse_proxy \[::1\]:8000 \{/, 'caddy: loopback upstream with the port');
like($conf, qr|proxy_pass http://\[::1\]:8000;|, 'nginx: loopback upstream with the port'); like($conf, qr/flush_interval -1/, 'caddy: unbuffered streaming');
like($conf, qr/proxy_http_version 1\.1;/, 'nginx: HTTP/1.1 for streaming'); like($conf, qr/header_up X-Real-IP \{remote_host\}/, 'caddy: X-Real-IP survives the heredoc');
like($conf, qr/proxy_buffering off;/, 'nginx: buffering off for streaming'); like($conf, qr/max_size 50MB/, 'caddy: the request body limit');
like($conf, qr/proxy_set_header Host \$host;/, 'nginx: $host survives the heredoc'); like($conf, qr/\treverse_proxy /, 'caddy: tab-indented as the Caddyfile is formatted');
like($conf, qr/proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;/, check_unlike($conf, qr/\bbind\b/, 'caddy: no bind directive, the kernel decides the families');
'nginx: $proxy_add_x_forwarded_for survives the heredoc'); check_unlike($conf, qr/acme|on_demand|http:\/\/\{/i, 'caddy: no ACME, the self-signed pair is used');
my $conf4 = nginx_conf_content(8000, '127.0.0.1', '/etc/ssl/sglang'); my $conf4 = caddyfile_content(9000, '127.0.0.1', '/etc/ssl/llm');
is($conf4 =~ /\[::\]/ ? 'yes' : 'no', like($conf4, qr/reverse_proxy 127\.0\.0\.1:9000 \{/, 'caddy: an IPv4 upstream carries the port');
(-e '/proc/net/if_inet6' ? 'yes' : 'no'), 'nginx: IPv6 listener follows the kernel'); like($conf4, qr|tls /etc/ssl/llm/sglang\.crt|, 'caddy: the certificate directory follows --cert-dir');
is(caddyfile_path('sglang'), '/etc/caddy/Caddyfile.d/sglang.caddyfile',
'caddy: the drop-in path follows the service name');
is(caddy_main_config(), '/etc/caddy/Caddyfile', 'caddy: the main Caddyfile path');
# ---- caddy release binary -------------------------------------------------
is(uname_to_arch('x86_64'), 'amd64', 'caddy binary: x86_64 maps to amd64');
is(uname_to_arch('aarch64'), 'arm64', 'caddy binary: aarch64 maps to arm64');
is(uname_to_arch('ppc64le'), undef, 'caddy binary: an unmapped machine is refused');
is(caddy_asset_url('2.10.2', 'amd64'),
'https://github.com/caddyserver/caddy/releases/download/v2.10.2/caddy_2.10.2_linux_amd64.tar.gz',
'caddy binary: the release asset URL');
my $caddy_unit = caddy_unit_content();
like($caddy_unit, qr|ExecStartPre=/usr/local/bin/caddy validate --config /etc/caddy/Caddyfile|,
'caddy binary: the unit validates before it starts');
like($caddy_unit, qr|ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile|,
'caddy binary: the unit runs the release binary');
like($caddy_unit, qr|ExecReload=/usr/local/bin/caddy reload --config /etc/caddy/Caddyfile|,
'caddy binary: the unit reloads through the admin endpoint');
like($caddy_unit, qr/^User=caddy$/m, 'caddy binary: the unit runs as the caddy user');
like($caddy_unit, qr/AmbientCapabilities=CAP_NET_BIND_SERVICE/, 'caddy binary: the port capability');
# ---- systemd unit -------------------------------------------------------- # ---- systemd unit --------------------------------------------------------
my $unit = systemd_content({ my $unit = systemd_content({