Compare commits
3
Commits
main
..
development
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
108a166ee0 | ||
|
|
37e06d090c | ||
|
|
9455c65f10 |
@@ -22,7 +22,7 @@ own builtins, and none of them needs a module installed beside it.
|
|||||||
automatic updates); `workstation-setup.pl` sets up a Fedora desktop (Brave, the
|
automatic updates); `workstation-setup.pl` sets up a Fedora desktop (Brave, the
|
||||||
official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux).
|
official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux).
|
||||||
- **Deployment**: `sglang-deploy.pl` puts an SGLang inference server on an AMD GPU
|
- **Deployment**: `sglang-deploy.pl` puts an SGLang inference server on an AMD GPU
|
||||||
behind nginx with HTTPS and an API key, runs the engine from the project's ROCm
|
behind Caddy with HTTPS and an API key, runs the engine from the project's ROCm
|
||||||
container image, and downloads the model weights from ModelScope.
|
container image, and downloads the model weights from ModelScope.
|
||||||
- **Maintenance**: `system-optimise.pl` removes old kernels (the running one and one
|
- **Maintenance**: `system-optimise.pl` removes old kernels (the running one and one
|
||||||
fallback always stay), vacuums journals, clears temporary files and core dumps, and
|
fallback always stay), vacuums journals, clears temporary files and core dumps, and
|
||||||
@@ -90,9 +90,9 @@ a change would do without doing it.
|
|||||||
| `network-diag.pl` | 2.0.0 | Network latency, DNS, MTU, packet loss and dual-stack on Linux and FreeBSD |
|
| `network-diag.pl` | 2.0.0 | Network latency, DNS, MTU, packet loss and dual-stack on Linux and FreeBSD |
|
||||||
| `system-diag.pl` | 2.0.0 | System health with a grade from A to F. Linux only |
|
| `system-diag.pl` | 2.0.0 | System health with a grade from A to F. Linux only |
|
||||||
| `security-audit.pl` | 2.0.0 | Security posture with a grade from A to F and an exit code for cron. Linux only |
|
| `security-audit.pl` | 2.0.0 | Security posture with a grade from A to F and an exit code for cron. Linux only |
|
||||||
| `server-setup.pl` | 2.0.0 | Server initial setup for Fedora, CentOS Stream and openEuler |
|
| `server-setup.pl` | 2.1.0 | Server initial setup for Fedora, CentOS Stream and openEuler |
|
||||||
| `workstation-setup.pl` | 2.0.0 | Fedora desktop setup |
|
| `workstation-setup.pl` | 2.0.0 | Fedora desktop setup |
|
||||||
| `sglang-deploy.pl` | 2.0.0 | SGLang inference server on an AMD GPU, behind nginx with HTTPS |
|
| `sglang-deploy.pl` | 2.1.0 | SGLang inference server on an AMD GPU, behind Caddy with HTTPS |
|
||||||
| `system-optimise.pl` | 2.0.0 | System cleanup; refuses rpm-ostree systems |
|
| `system-optimise.pl` | 2.0.0 | System cleanup; refuses rpm-ostree systems |
|
||||||
|
|
||||||
The full flag reference is in [docs/CLI.md](docs/CLI.md).
|
The full flag reference is in [docs/CLI.md](docs/CLI.md).
|
||||||
|
|||||||
+1
-1
@@ -61,7 +61,7 @@ is a defect worth reporting:
|
|||||||
user who runs the script.
|
user who runs the script.
|
||||||
- Missing hardening with no demonstrated impact, such as a script not setting a stricter
|
- Missing hardening with no demonstrated impact, such as a script not setting a stricter
|
||||||
umask than the system's.
|
umask than the system's.
|
||||||
- Flaws in a third-party tool the scripts drive (`dnf`, `podman`, `openssl`, `nginx` and
|
- Flaws in a third-party tool the scripts drive (`dnf`, `podman`, `openssl`, `caddy` and
|
||||||
the rest): report those to that project, and here only when this repository's use of
|
the rest): report those to that project, and here only when this repository's use of
|
||||||
the tool makes the flaw reachable in a way the tool's own documentation does not
|
the tool makes the flaw reachable in a way the tool's own documentation does not
|
||||||
anticipate.
|
anticipate.
|
||||||
|
|||||||
+15
-3
@@ -37,7 +37,7 @@ reports everything as already in place and changes nothing.
|
|||||||
| `system-diag.pl` | Reading CPU, memory, disk, network, GPU, services, security and performance, and grading the result | Change anything on the host; it is read-only |
|
| `system-diag.pl` | Reading CPU, memory, disk, network, GPU, services, security and performance, and grading the result | Change anything on the host; it is read-only |
|
||||||
| `server-setup.pl` | Base packages, firewall (with the SSH rule verified before the service starts), SELinux, Podman, automatic updates | Install the services themselves; that is the operator's, and the deploy scripts' |
|
| `server-setup.pl` | Base packages, firewall (with the SSH rule verified before the service starts), SELinux, Podman, automatic updates | Install the services themselves; that is the operator's, and the deploy scripts' |
|
||||||
| `workstation-setup.pl` | A Fedora desktop: Brave, the official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux | Anything on a server distribution; it targets Fedora Workstation |
|
| `workstation-setup.pl` | A Fedora desktop: Brave, the official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux | Anything on a server distribution; it targets Fedora Workstation |
|
||||||
| `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, nginx with TLS, the API key, the firewall rule and the SELinux boolean | The host's own setup, which `server-setup.pl` does first |
|
| `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, Caddy with TLS, the API key, the firewall rule and the SELinux checks | The host's own setup, which `server-setup.pl` does first |
|
||||||
| `system-optimise.pl` | Old kernels (the running one and one fallback always stay), journals, temporary files, core dumps, and the package audit | Touch an rpm-ostree system, which it refuses |
|
| `system-optimise.pl` | Old kernels (the running one and one fallback always stay), journals, temporary files, core dumps, and the package audit | Touch an rpm-ostree system, which it refuses |
|
||||||
|
|
||||||
## Data flow: a forked section collection
|
## Data flow: a forked section collection
|
||||||
@@ -97,8 +97,8 @@ that CentOS Stream and openEuler cannot carry at all.
|
|||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
Client[client] -->|443| Nginx[nginx on the host, TLS]
|
Client[client] -->|443| Caddy[Caddy on the host, TLS]
|
||||||
Nginx -->|loopback, plain HTTP| Engine[SGLang in a Podman container]
|
Caddy -->|loopback, plain HTTP| Engine[SGLang in a Podman container]
|
||||||
Engine -->|device nodes| GPU[/dev/kfd, /dev/dri]
|
Engine -->|device nodes| GPU[/dev/kfd, /dev/dri]
|
||||||
Engine -->|bind mount| Cache[state directory, model cache]
|
Engine -->|bind mount| Cache[state directory, model cache]
|
||||||
Unit[systemd unit] -->|podman run| Engine
|
Unit[systemd unit] -->|podman run| Engine
|
||||||
@@ -114,12 +114,24 @@ instead, and `--image` pins one. Radeon cards need `SGLANG_USE_AITER=false` and
|
|||||||
`SGLANG_ROCM_FUSED_DECODE_MLA=false` in the unit, which the script writes for them and
|
`SGLANG_ROCM_FUSED_DECODE_MLA=false` in the unit, which the script writes for them and
|
||||||
never for an Instinct host.
|
never for an Instinct host.
|
||||||
|
|
||||||
|
Caddy serves the endpoint from a drop-in under `/etc/caddy/Caddyfile.d`, which the
|
||||||
|
distribution's default Caddyfile imports. Fedora carries the caddy package and CentOS
|
||||||
|
Stream gets it from EPEL, whose repository file the script installs first; openEuler
|
||||||
|
packages no caddy at all, so there the official release binary is installed instead,
|
||||||
|
with the unit file the package would have carried. The service runs as the caddy user,
|
||||||
|
so the private key is made group-readable for the caddy group, and on an
|
||||||
|
SELinux-enforcing host the packaged caddy runs unconfined: no boolean is needed. A
|
||||||
|
deployment made by an earlier release of this script carries an nginx configuration,
|
||||||
|
which both a deploy and an uninstall remove.
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|
||||||
Nothing outside the interpreter, and nothing that has to be installed beyond the tools
|
Nothing outside the interpreter, and nothing that has to be installed beyond the tools
|
||||||
each script's own dependency section installs. The non-obvious ones and their reasons:
|
each script's own dependency section installs. The non-obvious ones and their reasons:
|
||||||
|
|
||||||
- `podman` for `sglang-deploy.pl`, because the engine is a container.
|
- `podman` for `sglang-deploy.pl`, because the engine is a container.
|
||||||
|
- `caddy` for `sglang-deploy.pl`, because the endpoint serves TLS on 443; the
|
||||||
|
release binary and `tar` stand in where no repository carries the package.
|
||||||
- `lspci` for the GPU family, and `rocm-smi` in `system-diag.pl` for AMD memory and
|
- `lspci` for the GPU family, and `rocm-smi` in `system-diag.pl` for AMD memory and
|
||||||
utilisation figures.
|
utilisation figures.
|
||||||
- `sha256sum` in `workstation-setup.pl`, because Perl's builtins have no hash.
|
- `sha256sum` in `workstation-setup.pl`, because Perl's builtins have no hash.
|
||||||
|
|||||||
+5
-5
@@ -111,7 +111,7 @@ verified by checksum; Brave's repository key is verified by fingerprint before i
|
|||||||
```
|
```
|
||||||
Usage: sglang-deploy.pl [options]
|
Usage: sglang-deploy.pl [options]
|
||||||
|
|
||||||
--model ID Hugging Face model ID (menu when omitted)
|
--model ID ModelScope model ID (menu when omitted)
|
||||||
--port N internal engine port (default: 8000, not 443)
|
--port N internal engine port (default: 8000, not 443)
|
||||||
--tensor-parallel N GPUs for tensor parallelism (default: 1, written as
|
--tensor-parallel N GPUs for tensor parallelism (default: 1, written as
|
||||||
the engine's --tp-size)
|
the engine's --tp-size)
|
||||||
@@ -130,19 +130,19 @@ Usage: sglang-deploy.pl [options]
|
|||||||
--api-key KEY API key for the endpoint (default: generate and
|
--api-key KEY API key for the endpoint (default: generate and
|
||||||
store in /etc/sysconfig)
|
store in /etc/sysconfig)
|
||||||
--dry-run preview without making changes
|
--dry-run preview without making changes
|
||||||
--uninstall tear down the service, container, nginx config
|
--uninstall tear down the service, container, caddy drop-in
|
||||||
and certificates
|
and certificates
|
||||||
--help show this help
|
--help show this help
|
||||||
--version show the version
|
--version show the version
|
||||||
```
|
```
|
||||||
|
|
||||||
Needs root. Without `--model` an interactive menu offers GLM 5.3, GLM 5.3 Flash,
|
Needs root. Without `--model` an interactive menu offers GLM 5.3, GLM 5.3 Flash,
|
||||||
DeepSeek V4 Pro, DeepSeek V4 Flash, MiMo V2.5 Pro and MiMo V2.5, plus a free-form
|
Qwen 3.8 Max, Qwen 3.8 Flash and DeepSeek V4.1 Flash, plus a free-form entry.
|
||||||
entry. `--tensor-parallel`, `--max-model-len` and `--gpu-memory-utilization` are
|
`--tensor-parallel`, `--max-model-len` and `--gpu-memory-utilization` are
|
||||||
written to the unit as the engine's own `--tp-size`, `--context-length` and
|
written to the unit as the engine's own `--tp-size`, `--context-length` and
|
||||||
`--mem-fraction-static`. The API key is shown once when it is generated; a key given
|
`--mem-fraction-static`. The API key is shown once when it is generated; a key given
|
||||||
with `--api-key` is never echoed. `--uninstall` stops and disables the service, removes
|
with `--api-key` is never echoed. `--uninstall` stops and disables the service, removes
|
||||||
the unit, the container, the nginx configuration and the certificates, and keeps the
|
the unit, the container, the caddy drop-in and the certificates, and keeps the
|
||||||
image and the model cache.
|
image and the model cache.
|
||||||
|
|
||||||
## system-optimise.pl
|
## system-optimise.pl
|
||||||
|
|||||||
+3
-3
@@ -84,7 +84,7 @@ deployment an operator cares about, so the pipeline that publishes them never ru
|
|||||||
| Script | What it leaves behind | Where it is documented |
|
| Script | What it leaves behind | Where it is documented |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `server-setup.pl` | Packages, firewalld with the SSH rule already allowed, SELinux enforcing, Podman, unattended updates | [docs/CLI.md](CLI.md) |
|
| `server-setup.pl` | Packages, firewalld with the SSH rule already allowed, SELinux enforcing, Podman, unattended updates | [docs/CLI.md](CLI.md) |
|
||||||
| `sglang-deploy.pl` | A systemd unit running the engine in a Podman container, nginx with TLS in front, an API key file, the firewall rule and the SELinux boolean | [docs/ARCHITECTURE.md](ARCHITECTURE.md) |
|
| `sglang-deploy.pl` | A systemd unit running the engine in a Podman container, Caddy with TLS in front, an API key file, the firewall rule and the SELinux checks | [docs/ARCHITECTURE.md](ARCHITECTURE.md) |
|
||||||
| `workstation-setup.pl` | A Fedora desktop with the toolchains and applications installed | [docs/CLI.md](CLI.md) |
|
| `workstation-setup.pl` | A Fedora desktop with the toolchains and applications installed | [docs/CLI.md](CLI.md) |
|
||||||
|
|
||||||
The SGLang deployment is the only service in the collection, and the host needs no ROCm
|
The SGLang deployment is the only service in the collection, and the host needs no ROCm
|
||||||
@@ -100,6 +100,6 @@ where the script reads them:
|
|||||||
|
|
||||||
- The published copies are unversioned: whatever is on `main` is what is served.
|
- The published copies are unversioned: whatever is on `main` is what is served.
|
||||||
- `sglang-deploy.pl` keeps the engine's API key in `/etc/sysconfig/sglang`, mode 0600,
|
- `sglang-deploy.pl` keeps the engine's API key in `/etc/sysconfig/sglang`, mode 0600,
|
||||||
written by the script and never committed. A Hugging Face token for a gated model goes
|
written by the script and never committed. A ModelScope token for a gated model goes
|
||||||
into that same file as `HF_TOKEN`, which the unit forwards to the container.
|
into that same file as `MODELSCOPE_TOKEN`, which the unit forwards to the container.
|
||||||
- The deploy secrets live in the Gitea repository settings, under Actions, Secrets.
|
- The deploy secrets live in the Gitea repository settings, under Actions, Secrets.
|
||||||
|
|||||||
+9
-7
@@ -61,13 +61,15 @@ only be exercised as root.
|
|||||||
|
|
||||||
`tests/container/rig.pl` runs the real `sglang-deploy.pl` as root inside a container
|
`tests/container/rig.pl` runs the real `sglang-deploy.pl` as root inside a container
|
||||||
against a stub `PATH`: every command the script drives (`dnf`, `rpm`, `podman`,
|
against a stub `PATH`: every command the script drives (`dnf`, `rpm`, `podman`,
|
||||||
`systemctl`, `curl`, `openssl`, `nginx`, `firewall-cmd`, `getsebool`, `lspci`) is a
|
`systemctl`, `curl`, `openssl`, `caddy`, `tar`, `useradd`, `semodule`, `firewall-cmd`,
|
||||||
stub that answers from a fixture, so a run is deterministic and needs no network and no
|
`getsebool`, `lspci`) is a stub that answers from a fixture, so a run is deterministic
|
||||||
GPU. It covers the deploy end to end: the resolved image tag, the unit file, the nginx
|
and needs no network and no GPU. It covers the deploy end to end: the resolved image
|
||||||
configuration, the TLS certificate and its modes, the API key file, the SELinux
|
tag, the unit file, the caddy drop-in and the main Caddyfile's import, the release
|
||||||
boolean, the firewall rule, the idempotent second run, the dry run, the uninstall, the
|
binary install for the hosts without a caddy package, the EPEL bootstrap on CentOS
|
||||||
Radeon and MI300 paths, the offline and unpublished-tag failures, the argument
|
Stream, the legacy nginx clean-up, the TLS certificate and its modes, the API key file,
|
||||||
validation and the non-root refusal.
|
the SELinux decisions, the firewall rule, the idempotent second run, the dry run, the
|
||||||
|
uninstall, the Radeon and MI300 paths, the offline and unpublished-tag failures, the
|
||||||
|
argument validation and the non-root refusal.
|
||||||
|
|
||||||
Prepare the platform image once, since the base images carry no Perl:
|
Prepare the platform image once, since the base images carry no Perl:
|
||||||
|
|
||||||
|
|||||||
+43
-10
@@ -44,7 +44,7 @@
|
|||||||
use strict;
|
use strict;
|
||||||
use warnings;
|
use warnings;
|
||||||
|
|
||||||
my $VERSION = '2.0.0';
|
my $VERSION = '2.1.0';
|
||||||
|
|
||||||
my $BOLD = "\033[1m";
|
my $BOLD = "\033[1m";
|
||||||
my $RED = "\033[31m";
|
my $RED = "\033[31m";
|
||||||
@@ -67,16 +67,39 @@ my %SUPPORTED_OS = (
|
|||||||
);
|
);
|
||||||
|
|
||||||
# Base packages installed on Fedora, CentOS Stream and openEuler alike.
|
# Base packages installed on Fedora, CentOS Stream and openEuler alike.
|
||||||
|
# caddy is the reverse proxy the deploy scripts serve the endpoints through;
|
||||||
|
# openEuler ships no package for it, and install_packages says so and leaves
|
||||||
|
# it to the script that needs it.
|
||||||
my @BASE_PACKAGES = qw(
|
my @BASE_PACKAGES = qw(
|
||||||
nano curl wget htop tmux rsync nginx openssl jq fastfetch
|
nano curl wget htop tmux rsync caddy openssl jq fastfetch
|
||||||
);
|
);
|
||||||
|
|
||||||
# Services to open in firewalld by default. ssh is mandatory: losing it means
|
# Services to open in firewalld by default. ssh is mandatory: losing it means
|
||||||
# locking out remote administration.
|
# locking out remote administration.
|
||||||
my @FIREWALL_SERVICES = ('ssh');
|
my @FIREWALL_SERVICES = ('ssh');
|
||||||
|
|
||||||
# Opened in firewalld only when nginx is installed, which it is by default.
|
# Opened in firewalld only when caddy is installed, which it is by default
|
||||||
my @NGINX_FIREWALL_SERVICES = ('http', 'https');
|
# wherever the package exists.
|
||||||
|
my @CADDY_FIREWALL_SERVICES = ('http', 'https');
|
||||||
|
|
||||||
|
# The one base package a distribution's repositories do not carry, with the
|
||||||
|
# reason and who provides it instead.
|
||||||
|
my %UNPACKAGED = (
|
||||||
|
openeuler => {
|
||||||
|
caddy => 'openEuler ships no caddy package; the scripts that need the '
|
||||||
|
. 'proxy install the release binary themselves',
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
# Test-visible accessors: the catalogue is lexical to this file, so the checks
|
||||||
|
# under tests/ read the base package list and the unpackaged map through these.
|
||||||
|
sub base_packages { return @BASE_PACKAGES; }
|
||||||
|
|
||||||
|
sub unpackaged_for {
|
||||||
|
my ($os_id) = @_;
|
||||||
|
return () unless exists $UNPACKAGED{$os_id};
|
||||||
|
return %{ $UNPACKAGED{$os_id} };
|
||||||
|
}
|
||||||
|
|
||||||
# dnf-automatic configuration file. dnf 4 ships it with defaults; dnf 5 reads host
|
# dnf-automatic configuration file. dnf 4 ships it with defaults; dnf 5 reads host
|
||||||
# overrides from this path (its own defaults live in /usr/share/dnf5).
|
# overrides from this path (its own defaults live in /usr/share/dnf5).
|
||||||
@@ -565,19 +588,29 @@ sub ensure_epel {
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
sub install_packages {
|
sub install_packages {
|
||||||
my ($dry_run) = @_;
|
my ($os_id, $dry_run) = @_;
|
||||||
my (@installed, @skipped, @failed, @to_install);
|
my (@installed, @skipped, @failed, @to_install, @unpackaged);
|
||||||
|
|
||||||
_status('Checking base packages');
|
_status('Checking base packages');
|
||||||
for my $pkg (@BASE_PACKAGES) {
|
for my $pkg (@BASE_PACKAGES) {
|
||||||
|
# exists and not a bare lookup: descending into a missing os key
|
||||||
|
# would autovivify it.
|
||||||
|
if (exists $UNPACKAGED{$os_id} && exists $UNPACKAGED{$os_id}{$pkg}) {
|
||||||
|
push @unpackaged, [$pkg, $UNPACKAGED{$os_id}{$pkg}];
|
||||||
|
next;
|
||||||
|
}
|
||||||
if (rpm_installed($pkg)) { push @skipped, $pkg }
|
if (rpm_installed($pkg)) { push @skipped, $pkg }
|
||||||
else { push @to_install, $pkg }
|
else { push @to_install, $pkg }
|
||||||
}
|
}
|
||||||
|
|
||||||
my $already = scalar @skipped;
|
my $already = scalar @skipped;
|
||||||
my $missing = scalar @to_install;
|
my $missing = scalar @to_install;
|
||||||
my $total = scalar @BASE_PACKAGES;
|
my $total = scalar @BASE_PACKAGES - scalar @unpackaged;
|
||||||
_status_done("$already/$total already installed");
|
_status_done("$already/$total already installed");
|
||||||
|
for my $entry (@unpackaged) {
|
||||||
|
my ($pkg, $reason) = @$entry;
|
||||||
|
_info("$pkg is not packaged on this distribution: $reason");
|
||||||
|
}
|
||||||
|
|
||||||
if (!@to_install) {
|
if (!@to_install) {
|
||||||
_ok('All base packages already present');
|
_ok('All base packages already present');
|
||||||
@@ -663,9 +696,9 @@ sub setup_firewall {
|
|||||||
$info{installed} = 1;
|
$info{installed} = 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
# ssh is mandatory; http and https only when nginx is present.
|
# ssh is mandatory; http and https only when caddy is present.
|
||||||
my @services = @FIREWALL_SERVICES;
|
my @services = @FIREWALL_SERVICES;
|
||||||
push @services, @NGINX_FIREWALL_SERVICES if rpm_installed('nginx');
|
push @services, @CADDY_FIREWALL_SERVICES if rpm_installed('caddy');
|
||||||
|
|
||||||
# --- Permanent rules first, through the offline client, which needs no daemon
|
# --- Permanent rules first, through the offline client, which needs no daemon
|
||||||
my $permanent_changed = 0;
|
my $permanent_changed = 0;
|
||||||
@@ -1776,7 +1809,7 @@ sub main {
|
|||||||
# 3. Base packages
|
# 3. Base packages
|
||||||
print STDERR "\n${BOLD}── Base Packages ──$RESET\n";
|
print STDERR "\n${BOLD}── Base Packages ──$RESET\n";
|
||||||
if (!$opt{skip_packages}) {
|
if (!$opt{skip_packages}) {
|
||||||
$results{packages} = install_packages($opt{dry_run});
|
$results{packages} = install_packages($os_id, $opt{dry_run});
|
||||||
if (@{ $results{packages}{failed} }) {
|
if (@{ $results{packages}{failed} }) {
|
||||||
push @warnings, 'Some packages failed to install: '
|
push @warnings, 'Some packages failed to install: '
|
||||||
. join(', ', @{ $results{packages}{failed} });
|
. join(', ', @{ $results{packages}{failed} });
|
||||||
|
|||||||
+524
-141
@@ -4,12 +4,14 @@
|
|||||||
|
|
||||||
# Idempotent SGLang deployment for AMD ROCm GPUs.
|
# Idempotent SGLang deployment for AMD ROCm GPUs.
|
||||||
#
|
#
|
||||||
# SGLang behind nginx with self-signed TLS on Fedora, CentOS Stream or openEuler.
|
# SGLang behind Caddy with self-signed TLS on Fedora, CentOS Stream or openEuler.
|
||||||
# The engine binds to ::1 (IPv4 loopback fallback) on port 8000, internal only;
|
# The engine binds to ::1 (IPv4 loopback fallback) on port 8000, internal only;
|
||||||
# nginx proxies :443 to the loopback upstream with streaming (SSE) support. The
|
# Caddy proxies :443 to the loopback upstream and streams (SSE) unbuffered. The
|
||||||
# endpoint requires an API key, delivered to the service through a 0600
|
# endpoint requires an API key, delivered to the service through a 0600
|
||||||
# EnvironmentFile; nginx to engine proxying is allowed through SELinux on enforcing
|
# EnvironmentFile. Caddy's service runs as the caddy user, so the private key is
|
||||||
# systems.
|
# made group-readable for the caddy group, and on SELinux-enforcing hosts the
|
||||||
|
# distribution's caddy runs unconfined, which needs no boolean; where a confined
|
||||||
|
# caddy policy is loaded anyway the script sets httpd_can_network_connect.
|
||||||
#
|
#
|
||||||
# Why the engine runs in a container rather than straight on the host:
|
# Why the engine runs in a container rather than straight on the host:
|
||||||
#
|
#
|
||||||
@@ -19,8 +21,8 @@
|
|||||||
# Rust), which Fedora carries only partly and which CentOS Stream and openEuler, where
|
# Rust), which Fedora carries only partly and which CentOS Stream and openEuler, where
|
||||||
# ROCm itself is unsupported by AMD, cannot carry at all. Both AMD and SGLang document
|
# ROCm itself is unsupported by AMD, cannot carry at all. Both AMD and SGLang document
|
||||||
# the container as the way to run SGLang on ROCm, so the container is what this script
|
# the container as the way to run SGLang on ROCm, so the container is what this script
|
||||||
# deploys: podman runs the official image, and the host keeps nginx, TLS, the API key,
|
# deploys: podman runs the official image, and the host keeps Caddy, TLS, the API key,
|
||||||
# the firewall and the SELinux boolean. The host needs no ROCm userland, only the
|
# the firewall and the SELinux story. The host needs no ROCm userland, only the
|
||||||
# amdgpu kernel driver and its device nodes, /dev/kfd and /dev/dri.
|
# amdgpu kernel driver and its device nodes, /dev/kfd and /dev/dri.
|
||||||
#
|
#
|
||||||
# Radeon cards: the project publishes no stable image for gfx1151 (Strix Halo, the
|
# Radeon cards: the project publishes no stable image for gfx1151 (Strix Halo, the
|
||||||
@@ -42,7 +44,9 @@
|
|||||||
# keeps stdout and stderr apart in the scratch directory.
|
# keeps stdout and stderr apart in the scratch directory.
|
||||||
#
|
#
|
||||||
# External binaries used: dnf, rpm, curl, podman, lspci, openssl, systemctl,
|
# External binaries used: dnf, rpm, curl, podman, lspci, openssl, systemctl,
|
||||||
# getenforce, getsebool, setsebool, firewall-cmd and nginx.
|
# getenforce, getsebool, setsebool, semodule, firewall-cmd, caddy, tar, install and
|
||||||
|
# useradd (the last four only on a host where no repository carries the caddy
|
||||||
|
# package and the release binary is installed instead).
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# sglang-deploy.pl # interactive model selection
|
# sglang-deploy.pl # interactive model selection
|
||||||
@@ -55,7 +59,7 @@
|
|||||||
use strict;
|
use strict;
|
||||||
use warnings;
|
use warnings;
|
||||||
|
|
||||||
my $VERSION = '2.0.0';
|
my $VERSION = '2.1.0';
|
||||||
|
|
||||||
my $BOLD = "\033[1m";
|
my $BOLD = "\033[1m";
|
||||||
my $RED = "\033[31m";
|
my $RED = "\033[31m";
|
||||||
@@ -78,11 +82,11 @@ my %SUPPORTED_OS = (
|
|||||||
);
|
);
|
||||||
|
|
||||||
# Tools this script itself needs. podman is the engine's runtime: SGLang ships no
|
# Tools this script itself needs. podman is the engine's runtime: SGLang ships no
|
||||||
# ROCm wheel, so the server runs from the project's own ROCm image.
|
# ROCm wheel, so the server runs from the project's own ROCm image. caddy is
|
||||||
my @DNF_PACKAGES = qw(pciutils curl openssl podman);
|
# installed in its own step rather than in this transaction: a name the
|
||||||
|
# repositories do not carry aborts the whole dnf run, and openEuler ships no
|
||||||
# Packages the engine expects from server-setup.pl (warning only, not installed here).
|
# caddy at all (there the release binary takes its place).
|
||||||
my @REQUIRED_SERVER_PACKAGES = (['nginx', 'reverse proxy']);
|
my @DNF_PACKAGES = qw(pciutils curl openssl podman tar);
|
||||||
|
|
||||||
# Default models for interactive selection when --model is omitted, keyed by
|
# Default models for interactive selection when --model is omitted, keyed by
|
||||||
# display name. Every ID is a ModelScope repository, which is where the engine
|
# display name. Every ID is a ModelScope repository, which is where the engine
|
||||||
@@ -176,6 +180,19 @@ my $DEFAULT_CERT_DIR = '/etc/ssl/sglang';
|
|||||||
my $DEFAULT_SERVICE = 'sglang';
|
my $DEFAULT_SERVICE = 'sglang';
|
||||||
my $INTERNAL_PORT = 8000;
|
my $INTERNAL_PORT = 8000;
|
||||||
|
|
||||||
|
# Caddy, the endpoint's TLS proxy. Fedora carries the package, CentOS Stream
|
||||||
|
# gets it from EPEL, and openEuler ships none, so where no package can be
|
||||||
|
# installed the official release binary takes its place, with the unit file the
|
||||||
|
# package would have carried. The distribution's default Caddyfile imports the
|
||||||
|
# Caddyfile.d directory, which is the drop-in this script writes; a main file
|
||||||
|
# without the import line gets it appended.
|
||||||
|
my $CADDY_RELEASES_API = 'https://api.github.com/repos/caddyserver/caddy/releases/latest';
|
||||||
|
my $CADDY_FALLBACK_VERSION = '2.10.2';
|
||||||
|
my $CADDY_BINARY_PATH = '/usr/local/bin/caddy';
|
||||||
|
my $CADDY_CONFIG_DIR = '/etc/caddy';
|
||||||
|
my $CADDY_IMPORT_LINE = 'import Caddyfile.d/*.caddyfile';
|
||||||
|
my $LEGACY_NGINX_DIR = '/etc/nginx/conf.d';
|
||||||
|
|
||||||
# ModelScope model IDs look like "org/name", the same shape Hugging Face uses.
|
# ModelScope model IDs look like "org/name", the same shape Hugging Face uses.
|
||||||
# The strict pattern also keeps systemd specifier characters (%) and whitespace
|
# The strict pattern also keeps systemd specifier characters (%) and whitespace
|
||||||
# out of unit files.
|
# out of unit files.
|
||||||
@@ -287,7 +304,7 @@ sub write_file {
|
|||||||
my ($path, $content) = @_;
|
my ($path, $content) = @_;
|
||||||
open(my $fh, '>', $path) or return 0;
|
open(my $fh, '>', $path) or return 0;
|
||||||
# The flush of a buffered handle surfaces at close, so close is checked too:
|
# The flush of a buffered handle surfaces at close, so close is checked too:
|
||||||
# a full disk must not report a truncated unit file or nginx configuration
|
# a full disk must not report a truncated unit file or Caddyfile drop-in
|
||||||
# as written.
|
# as written.
|
||||||
my $ok = print {$fh} $content;
|
my $ok = print {$fh} $content;
|
||||||
$ok = 0 unless close($fh);
|
$ok = 0 unless close($fh);
|
||||||
@@ -691,7 +708,7 @@ sub ms_model_status {
|
|||||||
return 'unknown';
|
return 'unknown';
|
||||||
}
|
}
|
||||||
|
|
||||||
# Return (bind_host, nginx_upstream_host): IPv6 ::1 first.
|
# Return (bind_host, caddy_upstream_host): IPv6 ::1 first.
|
||||||
#
|
#
|
||||||
# Falls back to 127.0.0.1 on kernels with IPv6 disabled
|
# Falls back to 127.0.0.1 on kernels with IPv6 disabled
|
||||||
# (net.ipv6.conf.all.disable_ipv6=1), where binding ::1 would fail.
|
# (net.ipv6.conf.all.disable_ipv6=1), where binding ::1 would fail.
|
||||||
@@ -760,7 +777,7 @@ sub cert_san {
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
sub install_system_deps {
|
sub install_system_deps {
|
||||||
my ($dry_run) = @_;
|
my ($os_id, $dry_run) = @_;
|
||||||
my %results = (installed => [], skipped => [], failed => []);
|
my %results = (installed => [], skipped => [], failed => []);
|
||||||
|
|
||||||
my @missing;
|
my @missing;
|
||||||
@@ -820,17 +837,199 @@ sub install_system_deps {
|
|||||||
_fail('podman is not installed: the engine runs as a container and cannot start');
|
_fail('podman is not installed: the engine runs as a container and cannot start');
|
||||||
}
|
}
|
||||||
|
|
||||||
# Packages expected from server-setup.pl (warning only, not installed here).
|
# Caddy proxies the endpoint on 443. Fedora carries the package; CentOS
|
||||||
for my $entry (@REQUIRED_SERVER_PACKAGES) {
|
# Stream carries it in EPEL, whose repository file installs first; where no
|
||||||
my ($pkg, $purpose) = @$entry;
|
# repository carries it at all (openEuler ships none), the official release
|
||||||
if (!rpm_installed($pkg)) {
|
# binary takes its place, unit file included. The step is separate from the
|
||||||
_warn("$pkg ($purpose) is not installed: run server-setup.pl first");
|
# transaction above, because one unresolvable name aborts a whole dnf run.
|
||||||
|
my $caddy = caddy_binary();
|
||||||
|
if (defined $caddy) {
|
||||||
|
_status('Checking caddy');
|
||||||
|
my $result = run([$caddy, 'version'], timeout => 30);
|
||||||
|
my $version = $result->{out};
|
||||||
|
$version =~ s/^\s+//;
|
||||||
|
$version =~ s/\s+$//;
|
||||||
|
$version = (split /\s+/, $version)[0] // '';
|
||||||
|
_status_done($version ne '' ? $version : 'installed');
|
||||||
|
$results{caddy} = $version ne '' ? $version : 'installed';
|
||||||
|
}
|
||||||
|
elsif ($dry_run) {
|
||||||
|
_status('Checking caddy');
|
||||||
|
_status_done('would install');
|
||||||
|
$results{caddy} = 'dry run';
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
# CentOS Stream carries caddy in EPEL, and the repository file ships in
|
||||||
|
# its extras repository: installing it first is what makes caddy
|
||||||
|
# resolvable in the transaction below.
|
||||||
|
if ($os_id eq 'centos' && !rpm_installed('epel-release')) {
|
||||||
|
_status('Enabling EPEL (caddy is packaged there)');
|
||||||
|
my $epel = run(['dnf', 'install', '-y', 'epel-release'], timeout => $DNF_TIMEOUT);
|
||||||
|
if ($epel->{rc} == 0) {
|
||||||
|
_status_done('ok');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('failed');
|
||||||
|
my $err = $epel->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_info("dnf stderr: $err") if length $err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_status('Installing caddy');
|
||||||
|
my $package = run(['dnf', 'install', '-y', 'caddy'], timeout => $DNF_TIMEOUT);
|
||||||
|
if ($package->{rc} == 0) {
|
||||||
|
_status_done('package');
|
||||||
|
$results{caddy} = 'package';
|
||||||
|
}
|
||||||
|
elsif (install_caddy_binary()) {
|
||||||
|
_status_done('release binary');
|
||||||
|
$results{caddy} = 'release binary';
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('failed');
|
||||||
|
$results{caddy} = undef;
|
||||||
|
_fail('caddy is not available: the endpoint cannot be served on 443');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return \%results;
|
return \%results;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The caddy binary the script drives, package or release binary. An absolute
|
||||||
|
# fallback is needed because a systemd unit and a fresh install reach the
|
||||||
|
# binary before any PATH that carries /usr/local/bin.
|
||||||
|
sub caddy_binary {
|
||||||
|
my $exe = find_exe('caddy');
|
||||||
|
return $exe if defined $exe;
|
||||||
|
return (-f $CADDY_BINARY_PATH && -x _) ? $CADDY_BINARY_PATH : undef;
|
||||||
|
}
|
||||||
|
|
||||||
|
# The newest caddy release version ('2.10.2'), from the GitHub API with a
|
||||||
|
# constant as the fallback. The charset bound keeps whatever the API answers
|
||||||
|
# out of the download URL.
|
||||||
|
sub resolve_caddy_version {
|
||||||
|
my $listing = fetch_text($CADDY_RELEASES_API);
|
||||||
|
if ($listing =~ /"tag_name"\s*:\s*"v(\d+\.\d+\.\d+)"/) {
|
||||||
|
return $1;
|
||||||
|
}
|
||||||
|
return $CADDY_FALLBACK_VERSION;
|
||||||
|
}
|
||||||
|
|
||||||
|
# caddy publishes release assets as caddy_VERSION_linux_ARCH.tar.gz.
|
||||||
|
sub uname_to_arch {
|
||||||
|
my ($machine) = @_;
|
||||||
|
return 'amd64' if $machine eq 'x86_64';
|
||||||
|
return 'arm64' if $machine eq 'aarch64';
|
||||||
|
return undef;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub caddy_asset_url {
|
||||||
|
my ($version, $arch) = @_;
|
||||||
|
return "https://github.com/caddyserver/caddy/releases/download"
|
||||||
|
. "/v$version/caddy_${version}_linux_${arch}.tar.gz";
|
||||||
|
}
|
||||||
|
|
||||||
|
# Install caddy from the official release binary, for the hosts no repository
|
||||||
|
# carries the package for (openEuler ships none). Everything the package would
|
||||||
|
# have provided is provided here: the binary, the directories, the service user
|
||||||
|
# and the unit file, copied from the distribution's own unit. The caller owns
|
||||||
|
# the progress line; this reports only failures.
|
||||||
|
sub install_caddy_binary {
|
||||||
|
my $version = resolve_caddy_version();
|
||||||
|
my $machine = run(['uname', '-m'], timeout => 15)->{out};
|
||||||
|
$machine =~ s/^\s+//;
|
||||||
|
$machine =~ s/\s+$//;
|
||||||
|
my $arch = uname_to_arch($machine);
|
||||||
|
if (!defined $arch) {
|
||||||
|
_fail("caddy publishes no release binary for $machine");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
my $curl = find_exe('curl');
|
||||||
|
my $tar = find_exe('tar');
|
||||||
|
my $install = find_exe('install');
|
||||||
|
if (!defined $curl || !defined $tar || !defined $install) {
|
||||||
|
_fail('curl, tar or install is missing: cannot install the caddy release binary');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
my $dir = scratch_dir();
|
||||||
|
my $tarball = "$dir/caddy.tar.gz";
|
||||||
|
my $download = run([$curl, '-fsSL', '-o', $tarball, caddy_asset_url($version, $arch)],
|
||||||
|
timeout => 300);
|
||||||
|
if ($download->{rc} != 0) {
|
||||||
|
my $err = $download->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_fail("The caddy release download failed: $err");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
my $extract = "$dir/caddy-extract";
|
||||||
|
mkdir($extract, 0700);
|
||||||
|
my $unpacked = run([$tar, '-xzf', $tarball, '-C', $extract], timeout => 60);
|
||||||
|
if ($unpacked->{rc} != 0 || !-f "$extract/caddy") {
|
||||||
|
_fail('The caddy release archive is not readable');
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
for my $path ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d", '/var/lib/caddy') {
|
||||||
|
mkdir($path, 0755) unless -d $path;
|
||||||
|
}
|
||||||
|
my $placed = run([$install, '-m', '0755', "$extract/caddy", $CADDY_BINARY_PATH],
|
||||||
|
timeout => 30);
|
||||||
|
if ($placed->{rc} != 0) {
|
||||||
|
my $err = $placed->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_fail("Could not install $CADDY_BINARY_PATH: $err");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!getpwnam('caddy')) {
|
||||||
|
my $user = run(['useradd', '--system', '--home-dir', '/var/lib/caddy',
|
||||||
|
'--create-home', '--shell', '/sbin/nologin', 'caddy'], timeout => 30);
|
||||||
|
if ($user->{rc} != 0) {
|
||||||
|
_warn('The caddy user could not be created: create it before starting caddy');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
my $unit_path = '/etc/systemd/system/caddy.service';
|
||||||
|
if (!write_file($unit_path, caddy_unit_content())) {
|
||||||
|
_fail("Could not write $unit_path: " . os_error_text($unit_path));
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
run(['systemctl', 'daemon-reload'], timeout => 30);
|
||||||
|
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
# The unit file for a release-binary install: the distribution's own unit, with
|
||||||
|
# the binary path adjusted. validate in ExecStartPre is what keeps a broken
|
||||||
|
# Caddyfile from taking the service down at boot.
|
||||||
|
sub caddy_unit_content {
|
||||||
|
return <<"UNIT";
|
||||||
|
[Unit]
|
||||||
|
Description=Caddy web server
|
||||||
|
Documentation=https://caddyserver.com/docs/
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
User=caddy
|
||||||
|
Group=caddy
|
||||||
|
ExecStartPre=$CADDY_BINARY_PATH validate --config $CADDY_CONFIG_DIR/Caddyfile
|
||||||
|
ExecStart=$CADDY_BINARY_PATH run --environ --config $CADDY_CONFIG_DIR/Caddyfile
|
||||||
|
ExecReload=$CADDY_BINARY_PATH reload --config $CADDY_CONFIG_DIR/Caddyfile
|
||||||
|
TimeoutStopSec=5s
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
PrivateTmp=true
|
||||||
|
ProtectHome=true
|
||||||
|
ProtectSystem=full
|
||||||
|
AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
UNIT
|
||||||
|
}
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# 2. Pre-flight checks
|
# 2. Pre-flight checks
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -1119,6 +1318,21 @@ sub fetch_engine_image {
|
|||||||
# 5. TLS certificate (self-signed)
|
# 5. TLS certificate (self-signed)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# The caddy service runs as the caddy user (the package creates it), so the
|
||||||
|
# private key has to cross the group line: root keeps the ownership and the
|
||||||
|
# caddy group gets read. Without the group (the package is missing) the key
|
||||||
|
# stays root-only and the caddy step reports what is missing.
|
||||||
|
sub ensure_caddy_key_readable {
|
||||||
|
my ($key_path) = @_;
|
||||||
|
my ($group, undef, $gid) = getgrnam('caddy');
|
||||||
|
if (!defined $group) {
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
chown(0, $gid, $key_path);
|
||||||
|
chmod(0640, $key_path);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
sub setup_tls {
|
sub setup_tls {
|
||||||
my ($cert_dir, $dry_run) = @_;
|
my ($cert_dir, $dry_run) = @_;
|
||||||
my %results;
|
my %results;
|
||||||
@@ -1127,6 +1341,7 @@ sub setup_tls {
|
|||||||
|
|
||||||
_status('Checking TLS certificate');
|
_status('Checking TLS certificate');
|
||||||
if (-f $crt_path && -f $key_path) {
|
if (-f $crt_path && -f $key_path) {
|
||||||
|
$results{key_readable} = ensure_caddy_key_readable($key_path);
|
||||||
_status_done('already exists');
|
_status_done('already exists');
|
||||||
$results{cert_exists} = 1;
|
$results{cert_exists} = 1;
|
||||||
return \%results;
|
return \%results;
|
||||||
@@ -1172,6 +1387,7 @@ sub setup_tls {
|
|||||||
if ($result->{rc} == 0) {
|
if ($result->{rc} == 0) {
|
||||||
chmod 0600, $key_path;
|
chmod 0600, $key_path;
|
||||||
chmod 0644, $crt_path;
|
chmod 0644, $crt_path;
|
||||||
|
$results{key_readable} = ensure_caddy_key_readable($key_path);
|
||||||
_status_done('generated');
|
_status_done('generated');
|
||||||
$results{cert_created} = 1;
|
$results{cert_created} = 1;
|
||||||
}
|
}
|
||||||
@@ -1288,10 +1504,13 @@ sub encode_base64url {
|
|||||||
# 7. SELinux
|
# 7. SELinux
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
# Allow nginx to reach the engine's port on SELinux-enforcing systems.
|
# Allow Caddy to reach the engine's port on SELinux-enforcing systems.
|
||||||
#
|
#
|
||||||
# http_port_t covers 80/81/443/488/8008/8009/8443/9000 but not the engine's port, so
|
# The distributions package caddy without an SELinux policy module, so its
|
||||||
# on enforcing systems nginx needs httpd_can_network_connect.
|
# service runs unconfined and needs no boolean at all. Where a confined caddy
|
||||||
|
# policy is loaded anyway (a local module), proxying to the engine's port needs
|
||||||
|
# httpd_can_network_connect: http_port_t covers 80/81/443/488/8008/8009/8443/9000
|
||||||
|
# but not the engine's port.
|
||||||
sub setup_selinux {
|
sub setup_selinux {
|
||||||
my ($dry_run) = @_;
|
my ($dry_run) = @_;
|
||||||
my %results;
|
my %results;
|
||||||
@@ -1315,12 +1534,28 @@ sub setup_selinux {
|
|||||||
}
|
}
|
||||||
_status_done('enforcing');
|
_status_done('enforcing');
|
||||||
|
|
||||||
|
_status('Checking for a confined caddy policy');
|
||||||
|
my $semodule = find_exe('semodule');
|
||||||
|
my $confined = 0;
|
||||||
|
if (defined $semodule) {
|
||||||
|
my $list = run([$semodule, '-l'], timeout => 30);
|
||||||
|
# "semodule -l" lines read "100 caddy(pp)" or the plain "caddy 1.0"
|
||||||
|
# of older releases; the priority column is optional in the match.
|
||||||
|
$confined = 1 if $list->{rc} == 0 && $list->{out} =~ /^\s*(?:\d+\s+)?\S*caddy\b/m;
|
||||||
|
}
|
||||||
|
if (!$confined) {
|
||||||
|
_status_done('none (caddy runs unconfined)');
|
||||||
|
$results{selinux} = 'unconfined';
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
_status_done('confined policy loaded');
|
||||||
|
|
||||||
_status('Checking httpd_can_network_connect boolean');
|
_status('Checking httpd_can_network_connect boolean');
|
||||||
my $getsebool = find_exe('getsebool');
|
my $getsebool = find_exe('getsebool');
|
||||||
my $setsebool = find_exe('setsebool');
|
my $setsebool = find_exe('setsebool');
|
||||||
if (!defined $getsebool || !defined $setsebool) {
|
if (!defined $getsebool || !defined $setsebool) {
|
||||||
_status_done('tools missing');
|
_status_done('tools missing');
|
||||||
_warn('getsebool/setsebool not found: nginx proxying may be blocked (502)');
|
_warn('getsebool/setsebool not found: caddy proxying may be blocked (502)');
|
||||||
$results{selinux} = 'failed';
|
$results{selinux} = 'failed';
|
||||||
return \%results;
|
return \%results;
|
||||||
}
|
}
|
||||||
@@ -1356,59 +1591,143 @@ sub setup_selinux {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# 8. nginx configuration
|
# 8. Caddy configuration
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
# Return the nginx server block content.
|
# Return the Caddyfile drop-in for the endpoint.
|
||||||
#
|
#
|
||||||
# HTTP/1.1 with an empty Connection header and proxy_buffering off are required for
|
# Caddy streams proxied responses immediately when flush_interval is negative,
|
||||||
# the engine's SSE streaming: nginx's defaults (HTTP/1.0, buffering on) would hold a
|
# which the engine's SSE completions need; the nginx equivalent was HTTP/1.1
|
||||||
# whole streamed completion until generation finishes. The IPv6 listener is emitted
|
# with proxy_buffering off. Caddy sets X-Forwarded-For and X-Forwarded-Proto
|
||||||
# only when the kernel actually has IPv6 enabled (the same check as detect_loopback);
|
# itself and passes the Host header through, so only X-Real-IP is written.
|
||||||
# socket() on [::]:443 would otherwise fail with EAFNOSUPPORT and take nginx down.
|
# There is no read timeout: a completion that generates for minutes must not be
|
||||||
sub nginx_conf_content {
|
# cut at a fixed limit, and the response ends when the engine ends it. No bind
|
||||||
|
# directive is written: Caddy listens on both loopback families on kernels with
|
||||||
|
# IPv6 and falls back to IPv4 alone where the kernel has none. The nesting is
|
||||||
|
# tab-indented, which is how the Caddyfile is formatted.
|
||||||
|
sub caddyfile_content {
|
||||||
my ($port, $upstream_host, $cert_dir) = @_;
|
my ($port, $upstream_host, $cert_dir) = @_;
|
||||||
my $ipv6_listen = -e '/proc/net/if_inet6' ? "listen [::]:443 ssl;\n " : '';
|
|
||||||
return <<"CONF";
|
return <<"CONF";
|
||||||
server {
|
:443 {
|
||||||
${ipv6_listen}listen 443 ssl;
|
tls $cert_dir/$CONTAINER_NAME.crt $cert_dir/$CONTAINER_NAME.key
|
||||||
server_name _;
|
request_body {
|
||||||
|
max_size 50MB
|
||||||
ssl_certificate $cert_dir/$CONTAINER_NAME.crt;
|
}
|
||||||
ssl_certificate_key $cert_dir/$CONTAINER_NAME.key;
|
reverse_proxy $upstream_host:$port {
|
||||||
ssl_protocols TLSv1.2 TLSv1.3;
|
flush_interval -1
|
||||||
|
header_up X-Real-IP {remote_host}
|
||||||
client_max_body_size 50m;
|
}
|
||||||
|
|
||||||
location / {
|
|
||||||
proxy_pass http://$upstream_host:$port;
|
|
||||||
proxy_http_version 1.1;
|
|
||||||
proxy_set_header Connection "";
|
|
||||||
proxy_set_header Host \$host;
|
|
||||||
proxy_set_header X-Real-IP \$remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header X-Forwarded-Proto \$scheme;
|
|
||||||
proxy_buffering off;
|
|
||||||
proxy_read_timeout 300s;
|
|
||||||
proxy_send_timeout 300s;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
CONF
|
CONF
|
||||||
}
|
}
|
||||||
|
|
||||||
sub nginx_conf_path {
|
sub caddyfile_path {
|
||||||
my ($service_name) = @_;
|
my ($service_name) = @_;
|
||||||
return "/etc/nginx/conf.d/$service_name.conf";
|
return "$CADDY_CONFIG_DIR/Caddyfile.d/$service_name.caddyfile";
|
||||||
}
|
}
|
||||||
|
|
||||||
sub setup_nginx {
|
sub caddy_main_config {
|
||||||
|
return "$CADDY_CONFIG_DIR/Caddyfile";
|
||||||
|
}
|
||||||
|
|
||||||
|
# The main Caddyfile must import the drop-in directory. The distributions'
|
||||||
|
# default file carries the import; a host without the file gets a minimal one,
|
||||||
|
# and one that does not import gets the line appended, which is inert while the
|
||||||
|
# directory holds nothing else.
|
||||||
|
sub ensure_caddy_import {
|
||||||
|
my ($dry_run) = @_;
|
||||||
|
my %results;
|
||||||
|
my $main = caddy_main_config();
|
||||||
|
|
||||||
|
my $current = -f $main ? slurp($main) : '';
|
||||||
|
_status('Checking the Caddyfile import');
|
||||||
|
# Any import of the drop-in directory counts, not only this script's exact
|
||||||
|
# line: appending a second one would make Caddy read every drop-in twice.
|
||||||
|
if ($current =~ /^\s*import\s+Caddyfile\.d\//m) {
|
||||||
|
_status_done('present');
|
||||||
|
$results{caddy_import} = 'present';
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
if ($dry_run) {
|
||||||
|
_status_done('would add');
|
||||||
|
$results{caddy_import} = 'dry run';
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
for my $dir ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d") {
|
||||||
|
mkdir($dir, 0755) unless -d $dir;
|
||||||
|
}
|
||||||
|
my $desired = length($current)
|
||||||
|
? $current . (substr($current, -1) eq "\n" ? '' : "\n") . "$CADDY_IMPORT_LINE\n"
|
||||||
|
: "$CADDY_IMPORT_LINE\n";
|
||||||
|
if (write_file($main, $desired)) {
|
||||||
|
chmod 0644, $main;
|
||||||
|
_status_done(length $current ? 'added' : 'created');
|
||||||
|
$results{caddy_import} = length $current ? 'added' : 'created';
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('failed');
|
||||||
|
_fail("Could not write $main: " . os_error_text($main));
|
||||||
|
$results{caddy_import} = 0;
|
||||||
|
}
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
|
||||||
|
# A deployment made by the nginx release leaves its drop-in behind. Remove it,
|
||||||
|
# so exactly one proxy owns :443; nginx itself stays, for whatever else it serves.
|
||||||
|
sub remove_legacy_nginx {
|
||||||
|
my ($service_name, $dry_run) = @_;
|
||||||
|
my %results;
|
||||||
|
my $legacy = "$LEGACY_NGINX_DIR/$service_name.conf";
|
||||||
|
return \%results unless -f $legacy;
|
||||||
|
|
||||||
|
_status('Removing the legacy nginx configuration');
|
||||||
|
if ($dry_run) {
|
||||||
|
_status_done('dry run');
|
||||||
|
$results{legacy_nginx_removed} = 'dry run';
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
unlink($legacy);
|
||||||
|
if (systemctl_is_active('nginx')) {
|
||||||
|
my $reload = run(['systemctl', 'reload', 'nginx'], timeout => 30);
|
||||||
|
if ($reload->{rc} == 0) {
|
||||||
|
_status_done('removed and nginx reloaded');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('removed (nginx reload failed)');
|
||||||
|
my $err = $reload->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_warn("nginx reload failed: $err");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('removed (nginx not running)');
|
||||||
|
}
|
||||||
|
$results{legacy_nginx_removed} = 1;
|
||||||
|
return \%results;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub setup_caddy {
|
||||||
my ($port, $upstream_host, $cert_dir, $service_name, $dry_run) = @_;
|
my ($port, $upstream_host, $cert_dir, $service_name, $dry_run) = @_;
|
||||||
my %results;
|
my %results;
|
||||||
my $conf_path = nginx_conf_path($service_name);
|
|
||||||
my $desired_content = nginx_conf_content($port, $upstream_host, $cert_dir);
|
|
||||||
|
|
||||||
# Write the nginx config if it is missing or different.
|
my $import = ensure_caddy_import($dry_run);
|
||||||
_status('Checking nginx configuration');
|
$results{caddy_import} = $import->{caddy_import};
|
||||||
|
|
||||||
|
my $legacy = remove_legacy_nginx($service_name, $dry_run);
|
||||||
|
$results{legacy_nginx_removed} = $legacy->{legacy_nginx_removed}
|
||||||
|
if defined $legacy->{legacy_nginx_removed};
|
||||||
|
|
||||||
|
my $conf_path = caddyfile_path($service_name);
|
||||||
|
my $desired_content = caddyfile_content($port, $upstream_host, $cert_dir);
|
||||||
|
|
||||||
|
if (!$dry_run) {
|
||||||
|
for my $dir ($CADDY_CONFIG_DIR, "$CADDY_CONFIG_DIR/Caddyfile.d") {
|
||||||
|
mkdir($dir, 0755) unless -d $dir;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Write the drop-in if it is missing or different.
|
||||||
|
_status('Checking the caddy configuration');
|
||||||
if (-f $conf_path) {
|
if (-f $conf_path) {
|
||||||
my $current = slurp($conf_path);
|
my $current = slurp($conf_path);
|
||||||
$current =~ s/^\s+//;
|
$current =~ s/^\s+//;
|
||||||
@@ -1418,95 +1737,107 @@ sub setup_nginx {
|
|||||||
$desired =~ s/\s+$//;
|
$desired =~ s/\s+$//;
|
||||||
if ($current eq $desired) {
|
if ($current eq $desired) {
|
||||||
_status_done('already configured');
|
_status_done('already configured');
|
||||||
$results{nginx_configured} = 1;
|
$results{caddy_configured} = 1;
|
||||||
}
|
}
|
||||||
elsif ($dry_run) {
|
elsif ($dry_run) {
|
||||||
_status_done('would update');
|
_status_done('would update');
|
||||||
$results{nginx_configured} = 'dry run';
|
$results{caddy_configured} = 'dry run';
|
||||||
}
|
}
|
||||||
elsif (write_file($conf_path, $desired_content)) {
|
elsif (write_file($conf_path, $desired_content)) {
|
||||||
|
chmod 0644, $conf_path;
|
||||||
_status_done('updated');
|
_status_done('updated');
|
||||||
$results{nginx_configured} = 1;
|
$results{caddy_configured} = 1;
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('failed');
|
_status_done('failed');
|
||||||
_fail("Could not write $conf_path: " . os_error_text($conf_path));
|
_fail("Could not write $conf_path: " . os_error_text($conf_path));
|
||||||
$results{nginx_configured} = 0;
|
$results{caddy_configured} = 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
elsif ($dry_run) {
|
elsif ($dry_run) {
|
||||||
_status_done('would create');
|
_status_done('would create');
|
||||||
$results{nginx_configured} = 'dry run';
|
$results{caddy_configured} = 'dry run';
|
||||||
}
|
}
|
||||||
elsif (write_file($conf_path, $desired_content)) {
|
elsif (write_file($conf_path, $desired_content)) {
|
||||||
|
chmod 0644, $conf_path;
|
||||||
_status_done('created');
|
_status_done('created');
|
||||||
$results{nginx_configured} = 1;
|
$results{caddy_configured} = 1;
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('failed');
|
_status_done('failed');
|
||||||
_fail("Could not write $conf_path: " . os_error_text($conf_path));
|
_fail("Could not write $conf_path: " . os_error_text($conf_path));
|
||||||
$results{nginx_configured} = 0;
|
$results{caddy_configured} = 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
# Ensure nginx is enabled and running (handles the enabled-but-stopped case).
|
# Ensure caddy is enabled and running (handles the enabled-but-stopped case).
|
||||||
_status('Ensuring nginx service is enabled and running');
|
_status('Ensuring caddy service is enabled and running');
|
||||||
my $nginx_running = systemctl_is_active('nginx');
|
my $caddy_running = systemctl_is_active('caddy');
|
||||||
if (systemctl_is_enabled('nginx') && $nginx_running) {
|
if (systemctl_is_enabled('caddy') && $caddy_running) {
|
||||||
_status_done('running');
|
_status_done('running');
|
||||||
$results{nginx_running} = 1;
|
$results{caddy_running} = 1;
|
||||||
}
|
}
|
||||||
elsif ($dry_run) {
|
elsif ($dry_run) {
|
||||||
_status_done('dry run');
|
_status_done('dry run');
|
||||||
$results{nginx_running} = 'dry run';
|
$results{caddy_running} = 'dry run';
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
my $start_result = systemctl_is_enabled('nginx')
|
my $start_result = systemctl_is_enabled('caddy')
|
||||||
? run(['systemctl', 'start', 'nginx'], timeout => 30)
|
? run(['systemctl', 'start', 'caddy'], timeout => 30)
|
||||||
: run(['systemctl', 'enable', '--now', 'nginx'], timeout => 30);
|
: run(['systemctl', 'enable', '--now', 'caddy'], timeout => 30);
|
||||||
if ($start_result->{rc} == 0) {
|
if ($start_result->{rc} == 0) {
|
||||||
_status_done('enabled and started');
|
_status_done('enabled and started');
|
||||||
$results{nginx_running} = 1;
|
$results{caddy_running} = 1;
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('failed');
|
_status_done('failed');
|
||||||
my $err = $start_result->{err};
|
my $err = $start_result->{err};
|
||||||
$err =~ s/\s+$//;
|
$err =~ s/\s+$//;
|
||||||
_warn("Could not start nginx: $err");
|
_warn("Could not start caddy: $err");
|
||||||
$results{nginx_running} = 0;
|
$results{caddy_running} = 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
# Test and reload the configuration (only possible when nginx is running).
|
# Validate and reload the configuration (only possible when caddy is running).
|
||||||
_status('Reloading nginx configuration');
|
_status('Reloading caddy configuration');
|
||||||
if ($dry_run) {
|
if ($dry_run) {
|
||||||
_status_done('dry run');
|
_status_done('dry run');
|
||||||
$results{nginx_reloaded} = 'dry run';
|
$results{caddy_reloaded} = 'dry run';
|
||||||
}
|
}
|
||||||
elsif (!$results{nginx_running}) {
|
elsif (!$results{caddy_running}) {
|
||||||
_status_done('skipped (nginx not running)');
|
_status_done('skipped (caddy not running)');
|
||||||
$results{nginx_reloaded} = 0;
|
$results{caddy_reloaded} = 0;
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
my $test_result = run(['nginx', '-t'], timeout => 30);
|
my $caddy = caddy_binary();
|
||||||
if ($test_result->{rc} != 0) {
|
if (!defined $caddy) {
|
||||||
_status_done('config test failed');
|
_status_done('caddy not found');
|
||||||
my $err = $test_result->{err};
|
_fail('caddy is not installed: cannot validate the configuration');
|
||||||
$err =~ s/\s+$//;
|
$results{caddy_reloaded} = 0;
|
||||||
_fail("nginx -t failed: $err");
|
|
||||||
$results{nginx_reloaded} = 0;
|
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30);
|
my $validate = run([$caddy, 'validate', '--config', caddy_main_config()],
|
||||||
if ($reload_result->{rc} == 0) {
|
timeout => 60);
|
||||||
_status_done('reloaded');
|
if ($validate->{rc} != 0) {
|
||||||
$results{nginx_reloaded} = 1;
|
_status_done('config test failed');
|
||||||
|
my $err = $validate->{err} . $validate->{out};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
my $tail = length($err) > 500 ? substr($err, -500) : $err;
|
||||||
|
_fail("caddy validate failed: $tail");
|
||||||
|
$results{caddy_reloaded} = 0;
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('failed');
|
my $reload_result = run(['systemctl', 'reload', 'caddy'], timeout => 30);
|
||||||
my $err = $reload_result->{err};
|
if ($reload_result->{rc} == 0) {
|
||||||
$err =~ s/\s+$//;
|
_status_done('reloaded');
|
||||||
_fail("nginx reload failed: $err");
|
$results{caddy_reloaded} = 1;
|
||||||
$results{nginx_reloaded} = 0;
|
}
|
||||||
|
else {
|
||||||
|
_status_done('failed');
|
||||||
|
my $err = $reload_result->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_fail("caddy reload failed: $err");
|
||||||
|
$results{caddy_reloaded} = 0;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1885,31 +2216,69 @@ sub uninstall {
|
|||||||
$results{env_not_found} = 1;
|
$results{env_not_found} = 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
# Remove the nginx config.
|
# Remove the caddy drop-in. The main Caddyfile stays: it may carry sites
|
||||||
my $nginx_conf = nginx_conf_path($service_name);
|
# this deployment knows nothing about, and the import line is inert once
|
||||||
_status("Removing nginx $service_name configuration");
|
# the drop-in is gone.
|
||||||
if (-f $nginx_conf) {
|
my $caddy_conf = caddyfile_path($service_name);
|
||||||
|
_status("Removing the caddy $service_name drop-in");
|
||||||
|
if (-f $caddy_conf) {
|
||||||
if ($dry_run) {
|
if ($dry_run) {
|
||||||
_status_done('dry run');
|
_status_done('dry run');
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
unlink($nginx_conf);
|
unlink($caddy_conf);
|
||||||
my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30);
|
if (systemctl_is_active('caddy')) {
|
||||||
if ($reload_result->{rc} != 0) {
|
my $reload_result = run(['systemctl', 'reload', 'caddy'], timeout => 30);
|
||||||
_status_done('removed (nginx reload failed)');
|
if ($reload_result->{rc} != 0) {
|
||||||
my $err = $reload_result->{err};
|
_status_done('removed (caddy reload failed)');
|
||||||
$err =~ s/\s+$//;
|
my $err = $reload_result->{err};
|
||||||
_warn("nginx reload failed: $err");
|
$err =~ s/\s+$//;
|
||||||
|
_warn("caddy reload failed: $err");
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('removed and caddy reloaded');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('removed and nginx reloaded');
|
_status_done('removed (caddy not running)');
|
||||||
}
|
}
|
||||||
$results{nginx_removed} = 1;
|
$results{caddy_removed} = 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('not present');
|
||||||
|
$results{caddy_not_found} = 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Remove the drop-in the nginx release wrote, when one is left over.
|
||||||
|
my $legacy_conf = "$LEGACY_NGINX_DIR/$service_name.conf";
|
||||||
|
_status('Removing the legacy nginx configuration');
|
||||||
|
if (-f $legacy_conf) {
|
||||||
|
if ($dry_run) {
|
||||||
|
_status_done('dry run');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
unlink($legacy_conf);
|
||||||
|
if (systemctl_is_active('nginx')) {
|
||||||
|
my $reload_result = run(['systemctl', 'reload', 'nginx'], timeout => 30);
|
||||||
|
if ($reload_result->{rc} != 0) {
|
||||||
|
_status_done('removed (nginx reload failed)');
|
||||||
|
my $err = $reload_result->{err};
|
||||||
|
$err =~ s/\s+$//;
|
||||||
|
_warn("nginx reload failed: $err");
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('removed and nginx reloaded');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_status_done('removed (nginx not running)');
|
||||||
|
}
|
||||||
|
$results{legacy_nginx_removed} = 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
_status_done('not present');
|
_status_done('not present');
|
||||||
$results{nginx_not_found} = 1;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# Remove the TLS certificates.
|
# Remove the TLS certificates.
|
||||||
@@ -1939,7 +2308,9 @@ sub uninstall {
|
|||||||
_info('Kept on the system (remove manually if unwanted):');
|
_info('Kept on the system (remove manually if unwanted):');
|
||||||
_info(" the engine image (podman rmi <image>)");
|
_info(" the engine image (podman rmi <image>)");
|
||||||
_info(" $state_dir (ModelScope cache with downloaded model weights)");
|
_info(" $state_dir (ModelScope cache with downloaded model weights)");
|
||||||
_info(" firewalld 'https' rule and the SELinux httpd_can_network_connect boolean");
|
_info(' the caddy service and /etc/caddy');
|
||||||
|
_info(" the firewalld 'https' rule (and the SELinux boolean, where a confined "
|
||||||
|
. 'caddy policy needed it)');
|
||||||
|
|
||||||
return \%results;
|
return \%results;
|
||||||
}
|
}
|
||||||
@@ -1966,7 +2337,8 @@ sub print_summary {
|
|||||||
['unit_removed', 'systemd unit removed'],
|
['unit_removed', 'systemd unit removed'],
|
||||||
['container_removed', 'engine container removed'],
|
['container_removed', 'engine container removed'],
|
||||||
['env_removed', 'API key environment file removed'],
|
['env_removed', 'API key environment file removed'],
|
||||||
['nginx_removed', 'nginx config removed'],
|
['caddy_removed', 'caddy drop-in removed'],
|
||||||
|
['legacy_nginx_removed', 'legacy nginx configuration removed'],
|
||||||
['certs_removed', 'TLS certificates removed'],
|
['certs_removed', 'TLS certificates removed'],
|
||||||
) {
|
) {
|
||||||
my ($key, $label) = @$pair;
|
my ($key, $label) = @$pair;
|
||||||
@@ -1981,7 +2353,7 @@ sub print_summary {
|
|||||||
['unit_not_found', 'systemd unit: already absent'],
|
['unit_not_found', 'systemd unit: already absent'],
|
||||||
['container_not_found', 'engine container: already absent'],
|
['container_not_found', 'engine container: already absent'],
|
||||||
['env_not_found', 'API key environment file: already absent'],
|
['env_not_found', 'API key environment file: already absent'],
|
||||||
['nginx_not_found', 'nginx config: already absent'],
|
['caddy_not_found', 'caddy drop-in: already absent'],
|
||||||
['certs_not_found', 'TLS certs: already absent'],
|
['certs_not_found', 'TLS certs: already absent'],
|
||||||
) {
|
) {
|
||||||
my ($key, $label) = @$pair;
|
my ($key, $label) = @$pair;
|
||||||
@@ -2096,17 +2468,28 @@ sub print_summary {
|
|||||||
elsif ($se && $se eq 'absent') {
|
elsif ($se && $se eq 'absent') {
|
||||||
_info('SELinux: not installed (skipped)');
|
_info('SELinux: not installed (skipped)');
|
||||||
}
|
}
|
||||||
|
elsif ($se && $se eq 'unconfined') {
|
||||||
|
_info('SELinux: caddy runs unconfined (nothing to do)');
|
||||||
|
}
|
||||||
|
|
||||||
my $ng = $results->{nginx} // {};
|
my $cd = $results->{caddy} // {};
|
||||||
if ($ng->{nginx_configured} && $ng->{nginx_configured} eq 1
|
if (defined $cd->{legacy_nginx_removed}) {
|
||||||
&& $ng->{nginx_reloaded} && $ng->{nginx_reloaded} eq 1) {
|
if ($cd->{legacy_nginx_removed} eq 1) {
|
||||||
_ok('nginx: configured and reloaded');
|
_ok('Legacy nginx configuration: removed');
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
_info('Legacy nginx configuration: would be removed');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
elsif ($ng->{nginx_configured} && $ng->{nginx_configured} eq 1) {
|
if ($cd->{caddy_configured} && $cd->{caddy_configured} eq 1
|
||||||
_fail('nginx: config written but reload failed');
|
&& $cd->{caddy_reloaded} && $cd->{caddy_reloaded} eq 1) {
|
||||||
|
_ok('Caddy: configured and reloaded');
|
||||||
}
|
}
|
||||||
elsif (($ng->{nginx_configured} // '') eq 'dry run') {
|
elsif ($cd->{caddy_configured} && $cd->{caddy_configured} eq 1) {
|
||||||
_info('nginx: would write config and reload');
|
_fail('Caddy: drop-in written but reload failed');
|
||||||
|
}
|
||||||
|
elsif (($cd->{caddy_configured} // '') eq 'dry run') {
|
||||||
|
_info('Caddy: would write the drop-in and reload');
|
||||||
}
|
}
|
||||||
|
|
||||||
my $svc = $results->{systemd} // {};
|
my $svc = $results->{systemd} // {};
|
||||||
@@ -2178,7 +2561,7 @@ Usage: sglang-deploy.pl [options]
|
|||||||
--api-key KEY API key for the endpoint (default: generate and
|
--api-key KEY API key for the endpoint (default: generate and
|
||||||
store in /etc/sysconfig)
|
store in /etc/sysconfig)
|
||||||
--dry-run preview without making changes
|
--dry-run preview without making changes
|
||||||
--uninstall tear down the service, container, nginx config
|
--uninstall tear down the service, container, caddy drop-in
|
||||||
and certificates
|
and certificates
|
||||||
--help show this help
|
--help show this help
|
||||||
--version show the version
|
--version show the version
|
||||||
@@ -2340,9 +2723,9 @@ sub is_positive_int {
|
|||||||
return defined $value && $value =~ /^\d+$/ && $value + 0 > 0;
|
return defined $value && $value =~ /^\d+$/ && $value + 0 > 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
# A directory the generated nginx configuration and the unit file carry
|
# A directory the generated Caddyfile drop-in and the unit file carry verbatim:
|
||||||
# verbatim: absolute, and free of the whitespace that splits arguments, of the
|
# absolute, and free of the whitespace that splits arguments and of the %
|
||||||
# % systemd expands as a specifier and of the ; that ends an nginx directive.
|
# systemd expands as a specifier.
|
||||||
sub valid_dir_path {
|
sub valid_dir_path {
|
||||||
my ($path) = @_;
|
my ($path) = @_;
|
||||||
return 0 unless defined $path && length $path;
|
return 0 unless defined $path && length $path;
|
||||||
@@ -2387,7 +2770,7 @@ sub validate_args {
|
|||||||
if (!is_positive_int($args->{port}) || $args->{port} + 0 > 65535
|
if (!is_positive_int($args->{port}) || $args->{port} + 0 > 65535
|
||||||
|| $args->{port} + 0 == 443) {
|
|| $args->{port} + 0 == 443) {
|
||||||
_fail("Invalid --port $args->{port}: must be an integer 1-65535 and not 443 "
|
_fail("Invalid --port $args->{port}: must be an integer 1-65535 and not 443 "
|
||||||
. "(nginx)");
|
. '(Caddy serves 443)');
|
||||||
exit 1;
|
exit 1;
|
||||||
}
|
}
|
||||||
if (!is_number($args->{gpu_memory_utilization})
|
if (!is_number($args->{gpu_memory_utilization})
|
||||||
@@ -2469,7 +2852,7 @@ sub main {
|
|||||||
# ── Deploy path ──
|
# ── Deploy path ──
|
||||||
# 1. System dependencies (before preflight: provides lspci, curl and podman).
|
# 1. System dependencies (before preflight: provides lspci, curl and podman).
|
||||||
print STDERR "\n${BOLD}── System Dependencies ──${RESET}\n";
|
print STDERR "\n${BOLD}── System Dependencies ──${RESET}\n";
|
||||||
$results{system_deps} = install_system_deps($args->{dry_run});
|
$results{system_deps} = install_system_deps($os_id, $args->{dry_run});
|
||||||
my @failed_pkgs = @{ $results{system_deps}{failed} // [] };
|
my @failed_pkgs = @{ $results{system_deps}{failed} // [] };
|
||||||
push @failures, 'failed to install packages: ' . join(', ', @failed_pkgs) if @failed_pkgs;
|
push @failures, 'failed to install packages: ' . join(', ', @failed_pkgs) if @failed_pkgs;
|
||||||
|
|
||||||
@@ -2501,7 +2884,7 @@ sub main {
|
|||||||
exit 1;
|
exit 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
# 5. TLS certificate (fatal, nginx cannot start without it).
|
# 5. TLS certificate (fatal, caddy cannot start without it).
|
||||||
print STDERR "\n${BOLD}── TLS Certificate ──${RESET}\n";
|
print STDERR "\n${BOLD}── TLS Certificate ──${RESET}\n";
|
||||||
$results{tls} = setup_tls($args->{cert_dir}, $args->{dry_run});
|
$results{tls} = setup_tls($args->{cert_dir}, $args->{dry_run});
|
||||||
if (defined $results{tls}{cert_created} && $results{tls}{cert_created} eq 0) {
|
if (defined $results{tls}{cert_created} && $results{tls}{cert_created} eq 0) {
|
||||||
@@ -2524,13 +2907,13 @@ sub main {
|
|||||||
push @failures, 'SELinux boolean httpd_can_network_connect not set';
|
push @failures, 'SELinux boolean httpd_can_network_connect not set';
|
||||||
}
|
}
|
||||||
|
|
||||||
# 8. nginx.
|
# 8. Caddy.
|
||||||
print STDERR "\n${BOLD}── nginx ──${RESET}\n";
|
print STDERR "\n${BOLD}── Caddy ──${RESET}\n";
|
||||||
$results{nginx} = setup_nginx(
|
$results{caddy} = setup_caddy(
|
||||||
$args->{port}, $upstream_host, $args->{cert_dir}, $args->{service_name}, $args->{dry_run},
|
$args->{port}, $upstream_host, $args->{cert_dir}, $args->{service_name}, $args->{dry_run},
|
||||||
);
|
);
|
||||||
if (defined $results{nginx}{nginx_reloaded} && $results{nginx}{nginx_reloaded} eq 0) {
|
if (defined $results{caddy}{caddy_reloaded} && $results{caddy}{caddy_reloaded} eq 0) {
|
||||||
push @failures, 'nginx configuration reload failed';
|
push @failures, 'caddy configuration reload failed';
|
||||||
}
|
}
|
||||||
|
|
||||||
# 9. systemd service (the model is probed before the unit is written).
|
# 9. systemd service (the model is probed before the unit is written).
|
||||||
|
|||||||
+188
-24
@@ -16,7 +16,11 @@ my $LOG = "$WORK/log/stubs.log";
|
|||||||
my $SCRIPT = $ENV{SGLANG_RIG_SCRIPT} // "$RIG/../../sglang-deploy.pl";
|
my $SCRIPT = $ENV{SGLANG_RIG_SCRIPT} // "$RIG/../../sglang-deploy.pl";
|
||||||
my $UNIT = '/etc/systemd/system/sglang.service';
|
my $UNIT = '/etc/systemd/system/sglang.service';
|
||||||
my $ENVFILE = '/etc/sysconfig/sglang';
|
my $ENVFILE = '/etc/sysconfig/sglang';
|
||||||
my $NGINX = '/etc/nginx/conf.d/sglang.conf';
|
my $CADDY = '/etc/caddy/Caddyfile.d/sglang.caddyfile';
|
||||||
|
my $CADDY_MAIN = '/etc/caddy/Caddyfile';
|
||||||
|
my $CADDY_UNIT = '/etc/systemd/system/caddy.service';
|
||||||
|
my $CADDY_BIN = '/usr/local/bin/caddy';
|
||||||
|
my $NGINX_LEGACY = '/etc/nginx/conf.d/sglang.conf';
|
||||||
my $CERTDIR = '/etc/ssl/sglang';
|
my $CERTDIR = '/etc/ssl/sglang';
|
||||||
my $STATEDIR = '/opt/sglang';
|
my $STATEDIR = '/opt/sglang';
|
||||||
|
|
||||||
@@ -105,22 +109,27 @@ sub mode_of {
|
|||||||
}
|
}
|
||||||
|
|
||||||
sub reset_fixture {
|
sub reset_fixture {
|
||||||
for my $path ($UNIT, $ENVFILE, $NGINX) {
|
for my $path ($UNIT, $ENVFILE, $CADDY, $CADDY_MAIN, $CADDY_UNIT, $CADDY_BIN,
|
||||||
|
$NGINX_LEGACY) {
|
||||||
unlink($path);
|
unlink($path);
|
||||||
}
|
}
|
||||||
|
remove_tree('/etc/caddy');
|
||||||
remove_tree($CERTDIR);
|
remove_tree($CERTDIR);
|
||||||
remove_tree($STATEDIR);
|
remove_tree($STATEDIR);
|
||||||
remove_tree($FIX);
|
remove_tree($FIX);
|
||||||
remove_tree($ST);
|
remove_tree($ST);
|
||||||
# Directories a host that ran server-setup.pl has: nginx's configuration drop-in
|
# Directories a host that ran server-setup.pl has: the legacy nginx drop-in
|
||||||
# and systemd's unit directory.
|
# directory an earlier release wrote into, and systemd's unit directory.
|
||||||
make_dirs($FIX, $ST, "$WORK/log", '/etc/nginx/conf.d', '/etc/systemd/system');
|
make_dirs($FIX, $ST, "$WORK/log", '/etc/nginx/conf.d', '/etc/systemd/system');
|
||||||
my $fh;
|
my $fh;
|
||||||
open($fh, q{>}, $LOG) and close($fh);
|
open($fh, q{>}, $LOG) and close($fh);
|
||||||
# Packages a real host or a previous run has already installed.
|
# Packages a real host or a previous run has already installed. nginx stands
|
||||||
|
# for the drop-in the previous release left behind.
|
||||||
write_fixture('rpm-installed', "nginx\n");
|
write_fixture('rpm-installed', "nginx\n");
|
||||||
write_fixture('fw-services', "\n");
|
write_fixture('fw-services', "\n");
|
||||||
# firewalld is running: server-setup.pl ensures it, and the firewall step needs it.
|
# firewalld is running: server-setup.pl ensures it, and the firewall step needs it.
|
||||||
|
# The handle is declared first: a my inside the open's argument list does not
|
||||||
|
# reach the right-hand operand of the and on this interpreter.
|
||||||
my $fw;
|
my $fw;
|
||||||
open($fw, '>', "$ST/active.firewalld") and close($fw);
|
open($fw, '>', "$ST/active.firewalld") and close($fw);
|
||||||
return;
|
return;
|
||||||
@@ -184,8 +193,8 @@ sub reset_log {
|
|||||||
# so the script's own PATH lookup finds them and nothing of the real system is used.
|
# so the script's own PATH lookup finds them and nothing of the real system is used.
|
||||||
sub prepare_stubs {
|
sub prepare_stubs {
|
||||||
make_dirs($BIN, $FIX, $ST, "$WORK/log");
|
make_dirs($BIN, $FIX, $ST, "$WORK/log");
|
||||||
for my $name (qw(lspci rpm dnf podman systemctl curl openssl nginx
|
for my $name (qw(lspci rpm dnf podman systemctl curl openssl caddy tar useradd
|
||||||
firewall-cmd getenforce getsebool setsebool)) {
|
semodule firewall-cmd getenforce getsebool setsebool)) {
|
||||||
my $link = "$BIN/$name";
|
my $link = "$BIN/$name";
|
||||||
# A link left over from a work directory that moved reads as broken to
|
# A link left over from a work directory that moved reads as broken to
|
||||||
# -e, and its stale target would leave the stubs unreachable: it is
|
# -e, and its stale target would leave the stubs unreachable: it is
|
||||||
@@ -197,6 +206,21 @@ sub prepare_stubs {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The caddy package creates its group; the rig creates it the same way, so the
|
||||||
|
# key permission the package makes possible is exercised for real. The group
|
||||||
|
# file is edited directly: the minimal openEuler image carries no groupadd to
|
||||||
|
# call, and a group entry is all the getgrnam in the script needs.
|
||||||
|
sub prepare_group {
|
||||||
|
return if defined getgrnam('caddy');
|
||||||
|
my $existing = slurp_file('/etc/group');
|
||||||
|
my $gid = 995;
|
||||||
|
$gid++ while $existing =~ /^[^:]+:[^:]*:\Q$gid\E:/m;
|
||||||
|
open(my $fh, '>>', '/etc/group') or die "cannot append to /etc/group: $!\n";
|
||||||
|
print {$fh} "caddy:x:$gid:\n";
|
||||||
|
close($fh);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
sub prepare_devices {
|
sub prepare_devices {
|
||||||
# The driver creates these on a real host; the rig fakes them (needs --privileged).
|
# The driver creates these on a real host; the rig fakes them (needs --privileged).
|
||||||
return if -e q{/dev/kfd};
|
return if -e q{/dev/kfd};
|
||||||
@@ -214,12 +238,15 @@ sub scenario_fresh {
|
|||||||
check($rc == 0, 'fresh: exit 0');
|
check($rc == 0, 'fresh: exit 0');
|
||||||
check(-f $UNIT, 'fresh: unit written');
|
check(-f $UNIT, 'fresh: unit written');
|
||||||
check(-f $ENVFILE, 'fresh: environment file written');
|
check(-f $ENVFILE, 'fresh: environment file written');
|
||||||
check(-f $NGINX, 'fresh: nginx configuration written');
|
check(-f $CADDY, 'fresh: caddy drop-in written');
|
||||||
|
check(-f $CADDY_MAIN, 'fresh: main Caddyfile written');
|
||||||
check(-f "$CERTDIR/sglang.crt" && -f "$CERTDIR/sglang.key", 'fresh: certificate written');
|
check(-f "$CERTDIR/sglang.crt" && -f "$CERTDIR/sglang.key", 'fresh: certificate written');
|
||||||
check(-d "$STATEDIR/modelscope", 'fresh: model cache directory created');
|
check(-d "$STATEDIR/modelscope", 'fresh: model cache directory created');
|
||||||
check(mode_of($ENVFILE) eq '0600', 'fresh: environment file is 0600 (' . mode_of($ENVFILE) . ')');
|
check(mode_of($ENVFILE) eq '0600', 'fresh: environment file is 0600 (' . mode_of($ENVFILE) . ')');
|
||||||
check(mode_of("$CERTDIR/sglang.key") eq '0600', 'fresh: key is 0600');
|
check(mode_of("$CERTDIR/sglang.key") eq '0640', 'fresh: key is 0640 for the caddy group (' . mode_of("$CERTDIR/sglang.key") . ')');
|
||||||
check(mode_of("$CERTDIR/sglang.crt") eq '0644', 'fresh: certificate is 0644');
|
check(mode_of("$CERTDIR/sglang.crt") eq '0644', 'fresh: certificate is 0644');
|
||||||
|
check((stat("$CERTDIR/sglang.key"))[5] == getgrnam('caddy'),
|
||||||
|
'fresh: the key belongs to the caddy group');
|
||||||
|
|
||||||
my $env = slurp_file($ENVFILE);
|
my $env = slurp_file($ENVFILE);
|
||||||
check_like($env, qr/^SGLANG_API_KEY=([A-Za-z0-9_-]{43})\n$/, 'fresh: generated key, url-safe, 43 chars');
|
check_like($env, qr/^SGLANG_API_KEY=([A-Za-z0-9_-]{43})\n$/, 'fresh: generated key, url-safe, 43 chars');
|
||||||
@@ -231,19 +258,29 @@ sub scenario_fresh {
|
|||||||
check_like($unit, qr/--mem-fraction-static 0\.9/, 'fresh: memory fraction default');
|
check_like($unit, qr/--mem-fraction-static 0\.9/, 'fresh: memory fraction default');
|
||||||
check_unlike($unit, qr/SGLANG_USE_AITER/, 'fresh: no Radeon variables on an Instinct host');
|
check_unlike($unit, qr/SGLANG_USE_AITER/, 'fresh: no Radeon variables on an Instinct host');
|
||||||
|
|
||||||
my $nginx = slurp_file($NGINX);
|
my $caddy = slurp_file($CADDY);
|
||||||
check_like($nginx, qr|proxy_pass http://\[::1\]:8000;|, 'fresh: nginx proxies to the loopback engine');
|
check_like($caddy, qr/reverse_proxy \[::1\]:8000 \{/, 'fresh: caddy proxies to the loopback engine');
|
||||||
check_like($nginx, qr|ssl_certificate /etc/ssl/sglang/sglang\.crt;|, 'fresh: nginx uses the sglang certificate');
|
check_like($caddy, qr|tls /etc/ssl/sglang/sglang\.crt /etc/ssl/sglang/sglang\.key|,
|
||||||
|
'fresh: caddy uses the sglang certificate pair');
|
||||||
|
check_like($caddy, qr/flush_interval -1/, 'fresh: streaming is unbuffered');
|
||||||
|
my $main = slurp_file($CADDY_MAIN);
|
||||||
|
check_like($main, qr/^import Caddyfile\.d\/\*\.caddyfile$/m, 'fresh: the main Caddyfile imports the drop-ins');
|
||||||
|
|
||||||
check(count_in_log(qr/^podman pull /) == 1, 'fresh: exactly one image pull');
|
check(count_in_log(qr/^podman pull /) == 1, 'fresh: exactly one image pull');
|
||||||
check_like(stub_log(), qr/^podman pull docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x$/m,
|
check_like(stub_log(), qr/^podman pull docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x$/m,
|
||||||
'fresh: the pulled image is the resolved tag');
|
'fresh: the pulled image is the resolved tag');
|
||||||
|
check_like(stub_log(), qr/^caddy version$/m, 'fresh: caddy is reported from the binary');
|
||||||
|
check_like(stub_log(), qr/^caddy validate --config \/etc\/caddy\/Caddyfile$/m,
|
||||||
|
'fresh: the configuration is validated before the reload');
|
||||||
|
check_like(stub_log(), qr/^systemctl enable --now caddy$/m, 'fresh: caddy enabled and started');
|
||||||
|
check_like(stub_log(), qr/^systemctl reload caddy$/m, 'fresh: caddy reloaded');
|
||||||
check(count_in_log(qr/^systemctl enable sglang$/) == 1, 'fresh: service enabled');
|
check(count_in_log(qr/^systemctl enable sglang$/) == 1, 'fresh: service enabled');
|
||||||
check(count_in_log(qr/^systemctl start sglang$/) == 1, 'fresh: service started');
|
check(count_in_log(qr/^systemctl start sglang$/) == 1, 'fresh: service started');
|
||||||
check_like(stub_log(), qr/^firewall-cmd --permanent --add-service=https$/m, 'fresh: HTTPS opened');
|
check_like(stub_log(), qr/^firewall-cmd --permanent --add-service=https$/m, 'fresh: HTTPS opened');
|
||||||
check_like($out, qr/Engine image: docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x/,
|
check_like($out, qr/Engine image: docker\.io\/lmsysorg\/sglang:v0\.5\.19-rocm724-mi30x/,
|
||||||
'fresh: summary names the image');
|
'fresh: summary names the image');
|
||||||
check_like($out, qr/systemd: sglang running on \[::1\]:8000/, 'fresh: summary names the endpoint');
|
check_like($out, qr/systemd: sglang running on \[::1\]:8000/, 'fresh: summary names the endpoint');
|
||||||
|
check_like($out, qr/Caddy: configured and reloaded/, 'fresh: summary reports caddy');
|
||||||
check_like($out, qr/API key \(shown once, store it securely\)/, 'fresh: the generated key is shown once');
|
check_like($out, qr/API key \(shown once, store it securely\)/, 'fresh: the generated key is shown once');
|
||||||
check_unlike($out, qr/✗/, 'fresh: no failed step');
|
check_unlike($out, qr/✗/, 'fresh: no failed step');
|
||||||
return;
|
return;
|
||||||
@@ -261,6 +298,7 @@ sub scenario_rerun {
|
|||||||
check_like(stub_log(), qr/^podman image exists /m, 'rerun: the image is checked instead');
|
check_like(stub_log(), qr/^podman image exists /m, 'rerun: the image is checked instead');
|
||||||
check(count_in_log(qr/^systemctl enable sglang$/) == 0, 'rerun: no second enable');
|
check(count_in_log(qr/^systemctl enable sglang$/) == 0, 'rerun: no second enable');
|
||||||
check(count_in_log(qr/^systemctl start sglang$/) == 0, 'rerun: no second start');
|
check(count_in_log(qr/^systemctl start sglang$/) == 0, 'rerun: no second start');
|
||||||
|
check(count_in_log(qr/^systemctl enable --now caddy$/) == 0, 'rerun: no second caddy enable');
|
||||||
check(count_in_log(qr/try-restart/) == 0, 'rerun: no restart, the unit did not change');
|
check(count_in_log(qr/try-restart/) == 0, 'rerun: no restart, the unit did not change');
|
||||||
check(count_in_log(qr/^dnf install /) == 0, 'rerun: no second package transaction');
|
check(count_in_log(qr/^dnf install /) == 0, 'rerun: no second package transaction');
|
||||||
check(slurp_file($ENVFILE) eq $key_before, 'rerun: the API key is reused, not regenerated');
|
check(slurp_file($ENVFILE) eq $key_before, 'rerun: the API key is reused, not regenerated');
|
||||||
@@ -282,7 +320,8 @@ sub scenario_dry_run {
|
|||||||
check($rc == 0, 'dry run: exit 0');
|
check($rc == 0, 'dry run: exit 0');
|
||||||
check(!-f $UNIT, 'dry run: no unit written');
|
check(!-f $UNIT, 'dry run: no unit written');
|
||||||
check(!-f $ENVFILE, 'dry run: no environment file written');
|
check(!-f $ENVFILE, 'dry run: no environment file written');
|
||||||
check(!-f $NGINX, 'dry run: no nginx configuration written');
|
check(!-f $CADDY, 'dry run: no caddy drop-in written');
|
||||||
|
check(!-e '/etc/caddy', 'dry run: no caddy directory created');
|
||||||
check(!-d $CERTDIR, 'dry run: no certificate directory');
|
check(!-d $CERTDIR, 'dry run: no certificate directory');
|
||||||
check(count_in_log(qr/^podman pull /) == 0, 'dry run: no pull');
|
check(count_in_log(qr/^podman pull /) == 0, 'dry run: no pull');
|
||||||
check(count_in_log(qr/^systemctl (enable|start) /) == 0, 'dry run: no service change');
|
check(count_in_log(qr/^systemctl (enable|start) /) == 0, 'dry run: no service change');
|
||||||
@@ -302,14 +341,17 @@ sub scenario_uninstall {
|
|||||||
check($rc == 0, 'uninstall: exit 0');
|
check($rc == 0, 'uninstall: exit 0');
|
||||||
check(!-f $UNIT, 'uninstall: unit removed');
|
check(!-f $UNIT, 'uninstall: unit removed');
|
||||||
check(!-f $ENVFILE, 'uninstall: environment file removed');
|
check(!-f $ENVFILE, 'uninstall: environment file removed');
|
||||||
check(!-f $NGINX, 'uninstall: nginx configuration removed');
|
check(!-f $CADDY, 'uninstall: caddy drop-in removed');
|
||||||
|
check(-f $CADDY_MAIN, 'uninstall: the main Caddyfile is kept');
|
||||||
check(!-d $CERTDIR, 'uninstall: certificate directory removed');
|
check(!-d $CERTDIR, 'uninstall: certificate directory removed');
|
||||||
check(-d $STATEDIR, 'uninstall: model cache kept');
|
check(-d $STATEDIR, 'uninstall: model cache kept');
|
||||||
check(-e "$ST/image.docker.io_lmsysorg_sglang_v0.5.19-rocm724-mi30x",
|
check(-e "$ST/image.docker.io_lmsysorg_sglang_v0.5.19-rocm724-mi30x",
|
||||||
'uninstall: the image is kept in podman');
|
'uninstall: the image is kept in podman');
|
||||||
check_like(stub_log(), qr/^systemctl stop sglang$/m, 'uninstall: service stopped');
|
check_like(stub_log(), qr/^systemctl stop sglang$/m, 'uninstall: service stopped');
|
||||||
check_like(stub_log(), qr/^systemctl disable sglang$/m, 'uninstall: service disabled');
|
check_like(stub_log(), qr/^systemctl disable sglang$/m, 'uninstall: service disabled');
|
||||||
|
check_like(stub_log(), qr/^systemctl reload caddy$/m, 'uninstall: caddy reloaded after the drop-in');
|
||||||
check_like($out, qr/SGLang service stopped/, 'uninstall: summary reports the stop');
|
check_like($out, qr/SGLang service stopped/, 'uninstall: summary reports the stop');
|
||||||
|
check_like($out, qr/caddy drop-in removed/, 'uninstall: summary reports the drop-in');
|
||||||
check_like($out, qr/Kept on the system/, 'uninstall: the kept state is listed');
|
check_like($out, qr/Kept on the system/, 'uninstall: the kept state is listed');
|
||||||
check_like($out, qr/ModelScope cache/, 'uninstall: the cache is named as kept');
|
check_like($out, qr/ModelScope cache/, 'uninstall: the cache is named as kept');
|
||||||
return;
|
return;
|
||||||
@@ -322,6 +364,78 @@ sub scenario_uninstall_twice {
|
|||||||
check($rc == 0, 'uninstall twice: exit 0');
|
check($rc == 0, 'uninstall twice: exit 0');
|
||||||
check_like($out, qr/already absent/, 'uninstall twice: idempotent');
|
check_like($out, qr/already absent/, 'uninstall twice: idempotent');
|
||||||
check(count_in_log(qr/^systemctl stop /) == 0, 'uninstall twice: nothing to stop');
|
check(count_in_log(qr/^systemctl stop /) == 0, 'uninstall twice: nothing to stop');
|
||||||
|
check(count_in_log(qr/^systemctl reload caddy$/) == 0,
|
||||||
|
'uninstall twice: no reload without a drop-in');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
# A host deployed by the nginx release carries its drop-in. Both a deploy and
|
||||||
|
# an uninstall remove it, so exactly one proxy owns :443 afterwards.
|
||||||
|
sub scenario_legacy_nginx {
|
||||||
|
reset_fixture();
|
||||||
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
|
open(my $fh, '>', $NGINX_LEGACY) or die "cannot write $NGINX_LEGACY: $!\n";
|
||||||
|
print {$fh} "server {\n listen 443 ssl;\n}\n";
|
||||||
|
close($fh);
|
||||||
|
# The stub state: nginx is running on this host, so the removal reloads it.
|
||||||
|
my $st;
|
||||||
|
open($st, '>', "$ST/active.nginx") and close($st);
|
||||||
|
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
|
check($rc == 0, 'legacy nginx: exit 0');
|
||||||
|
check(!-f $NGINX_LEGACY, 'legacy nginx: the old drop-in is gone on deploy');
|
||||||
|
check_like(stub_log(), qr/^systemctl reload nginx$/m,
|
||||||
|
'legacy nginx: the running nginx is reloaded');
|
||||||
|
check_like($out, qr/Legacy nginx configuration: removed/, 'legacy nginx: the summary names it');
|
||||||
|
|
||||||
|
# An uninstall on a host the new release never deployed cleans it too.
|
||||||
|
reset_fixture();
|
||||||
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
|
open($fh, '>', $NGINX_LEGACY) or die "cannot write $NGINX_LEGACY: $!\n";
|
||||||
|
print {$fh} "server {\n listen 443 ssl;\n}\n";
|
||||||
|
close($fh);
|
||||||
|
reset_log();
|
||||||
|
($rc, $out) = run_script('--uninstall');
|
||||||
|
check($rc == 0, 'legacy nginx: uninstall exit 0');
|
||||||
|
check(!-f $NGINX_LEGACY, 'legacy nginx: the old drop-in is gone on uninstall');
|
||||||
|
check_like($out, qr/legacy nginx configuration removed/, 'legacy nginx: the uninstall summary names it');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Where no repository carries the caddy package, the official release binary
|
||||||
|
# takes its place: download, extract, install, the service user, and the unit
|
||||||
|
# file the package would have carried.
|
||||||
|
sub scenario_caddy_binary {
|
||||||
|
reset_fixture();
|
||||||
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
|
write_fixture('caddy-no-package', "1\n");
|
||||||
|
unlink('/usr/bin/caddy'); # order independence: no package binary, no stub
|
||||||
|
unlink("$BIN/caddy") or die "cannot remove the caddy stub: $!\n";
|
||||||
|
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
|
|
||||||
|
check($rc == 0, 'caddy binary: exit 0');
|
||||||
|
check_like(stub_log(), qr/^dnf install -y caddy$/m, 'caddy binary: the package install was attempted');
|
||||||
|
check_like(stub_log(), qr{^curl -fsSL -o \S+ https://github\.com/caddyserver/caddy/releases/download/v2\.10\.2/caddy_2\.10\.2_linux_amd64\.tar\.gz$}m,
|
||||||
|
'caddy binary: the release asset is downloaded');
|
||||||
|
check_like(stub_log(), qr/^tar -xzf \S+ -C \S+$/m, 'caddy binary: the archive is extracted');
|
||||||
|
check(-x $CADDY_BIN, 'caddy binary: the binary is installed executable');
|
||||||
|
check_like(stub_log(), qr/^useradd --system --home-dir \/var\/lib\/caddy --create-home --shell \/sbin\/nologin caddy$/m,
|
||||||
|
'caddy binary: the service user is created');
|
||||||
|
check(-f $CADDY_UNIT, 'caddy binary: the unit file is written');
|
||||||
|
my $unit = slurp_file($CADDY_UNIT);
|
||||||
|
check_like($unit, qr|ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile|,
|
||||||
|
'caddy binary: the unit runs the release binary');
|
||||||
|
check_like(stub_log(), qr/^systemctl daemon-reload$/m, 'caddy binary: systemd reloaded');
|
||||||
|
check_like(stub_log(), qr{^caddy validate --config /etc/caddy/Caddyfile$}m,
|
||||||
|
'caddy binary: the installed binary validates');
|
||||||
|
check_like($out, qr/Caddy: configured and reloaded/, 'caddy binary: the deployment completes');
|
||||||
|
|
||||||
|
unlink($CADDY_BIN);
|
||||||
|
unlink($CADDY_UNIT);
|
||||||
|
unlink("$FIX/caddy-no-package");
|
||||||
|
symlink("$RIG/stub.pl", "$BIN/caddy") or die "cannot restore the caddy stub: $!\n";
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -495,7 +609,7 @@ sub scenario_validation {
|
|||||||
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--port', '443', '--dry-run');
|
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--port', '443', '--dry-run');
|
||||||
check($rc == 1, 'validation: port 443 refused');
|
check($rc == 1, 'validation: port 443 refused');
|
||||||
check_like($out, qr/must be 1-65535 and not 443/, 'validation: port message');
|
check_like($out, qr/must be an integer 1-65535 and not 443/, 'validation: port message');
|
||||||
($rc, $out) = run_script('--model', 'not-a-model-id', '--dry-run');
|
($rc, $out) = run_script('--model', 'not-a-model-id', '--dry-run');
|
||||||
check($rc == 1, 'validation: bad model ID refused');
|
check($rc == 1, 'validation: bad model ID refused');
|
||||||
check_like($out, qr/Invalid model ID/, 'validation: model message');
|
check_like($out, qr/Invalid model ID/, 'validation: model message');
|
||||||
@@ -545,7 +659,7 @@ sub scenario_non_root {
|
|||||||
my $rc = $? >> 8;
|
my $rc = $? >> 8;
|
||||||
my $out = slurp_file($out_file);
|
my $out = slurp_file($out_file);
|
||||||
check($rc == 0, 'non-root: --version works without root');
|
check($rc == 0, 'non-root: --version works without root');
|
||||||
check_like($out, qr/^sglang-deploy\.pl 2\.0\.0$/, 'non-root: the version is printed');
|
check_like($out, qr/^sglang-deploy\.pl 2\.1\.0$/, 'non-root: the version is printed');
|
||||||
|
|
||||||
my $pid3 = fork();
|
my $pid3 = fork();
|
||||||
die "cannot fork: $!\n" unless defined $pid3;
|
die "cannot fork: $!\n" unless defined $pid3;
|
||||||
@@ -583,12 +697,12 @@ sub scenario_dependency_section {
|
|||||||
reset_fixture();
|
reset_fixture();
|
||||||
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
write_fixture('rocm.txt', "7.2.4\n");
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
write_fixture('rpm-installed', "pciutils\ncurl\nopenssl\npodman\nnginx\n");
|
write_fixture('rpm-installed', "pciutils\ncurl\nopenssl\npodman\ntar\n");
|
||||||
reset_log();
|
reset_log();
|
||||||
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--dry-run');
|
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3', '--dry-run');
|
||||||
check($rc == 0, 'deps: exit 0');
|
check($rc == 0, 'deps: exit 0');
|
||||||
check(count_in_log(qr/^dnf install /) == 0, 'deps: no transaction when all packages are present');
|
check(count_in_log(qr/^dnf install /) == 0, 'deps: no transaction when all packages are present');
|
||||||
check_like($out, qr/All 4 packages already installed/, 'deps: reported as present');
|
check_like($out, qr/All 5 packages already installed/, 'deps: reported as present');
|
||||||
|
|
||||||
reset_fixture();
|
reset_fixture();
|
||||||
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
@@ -596,10 +710,41 @@ sub scenario_dependency_section {
|
|||||||
write_fixture('rpm-installed', "\n");
|
write_fixture('rpm-installed', "\n");
|
||||||
reset_log();
|
reset_log();
|
||||||
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
check_like(stub_log(), qr/^dnf install -y pciutils curl openssl podman$/m,
|
check_like(stub_log(), qr/^dnf install -y pciutils curl openssl podman tar$/m,
|
||||||
'deps: the four packages are installed in one transaction');
|
'deps: the five packages are installed in one transaction');
|
||||||
check_like($out, qr/nginx \(reverse proxy\) is not installed: run server-setup.pl first/,
|
check_like(stub_log(), qr/^caddy version$/m, 'deps: caddy is checked after the transaction');
|
||||||
'deps: the missing reverse proxy is warned about');
|
check_unlike($out, qr/run server-setup\.pl first/,
|
||||||
|
'deps: no warning points at server-setup.pl, caddy is installed here');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
# CentOS Stream carries caddy in EPEL, and the repository file installs first,
|
||||||
|
# which is what makes the package transaction below it resolvable.
|
||||||
|
sub scenario_epel {
|
||||||
|
reset_fixture();
|
||||||
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
|
my $real = slurp_file('/etc/os-release');
|
||||||
|
open(my $fh, '>', '/etc/os-release') or die "cannot write /etc/os-release: $!\n";
|
||||||
|
print {$fh} "ID=centos\nVERSION_ID=\"10\"\n";
|
||||||
|
close($fh);
|
||||||
|
unlink('/usr/bin/caddy'); # order independence: no binary, no stub
|
||||||
|
unlink("$BIN/caddy");
|
||||||
|
reset_log();
|
||||||
|
my ($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
|
open(my $back, '>', '/etc/os-release') or die "cannot restore /etc/os-release: $!\n";
|
||||||
|
print {$back} $real;
|
||||||
|
close($back);
|
||||||
|
|
||||||
|
check($rc == 0, 'epel: exit 0');
|
||||||
|
check_like(stub_log(), qr/^dnf install -y epel-release$/m, 'epel: the repository file installs first');
|
||||||
|
check_like(stub_log(), qr/^dnf install -y caddy$/m, 'epel: the package installs after it');
|
||||||
|
check_like($out, qr/Installing caddy\.\.\. package/, 'epel: the package path is taken');
|
||||||
|
check_like($out, qr/Caddy: configured and reloaded/, 'epel: the deployment completes');
|
||||||
|
|
||||||
|
unlink('/usr/bin/caddy');
|
||||||
|
unlink("$FIX/caddy-no-package");
|
||||||
|
symlink("$RIG/stub.pl", "$BIN/caddy") or die "cannot restore the caddy stub: $!\n";
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -619,7 +764,7 @@ sub scenario_custom_layout {
|
|||||||
);
|
);
|
||||||
check($rc == 0, 'custom layout: exit 0');
|
check($rc == 0, 'custom layout: exit 0');
|
||||||
check(-f '/etc/systemd/system/llm.service', 'custom layout: the named unit is written');
|
check(-f '/etc/systemd/system/llm.service', 'custom layout: the named unit is written');
|
||||||
check(-f '/etc/nginx/conf.d/llm.conf', 'custom layout: the named nginx file is written');
|
check(-f '/etc/caddy/Caddyfile.d/llm.caddyfile', 'custom layout: the named caddy drop-in is written');
|
||||||
# The certificate file names follow the program, as they did before, not the
|
# The certificate file names follow the program, as they did before, not the
|
||||||
# service name; the directory follows --cert-dir.
|
# service name; the directory follows --cert-dir.
|
||||||
check(-f '/etc/ssl/llm/sglang.crt', 'custom layout: certificates follow the directory');
|
check(-f '/etc/ssl/llm/sglang.crt', 'custom layout: certificates follow the directory');
|
||||||
@@ -632,7 +777,7 @@ sub scenario_custom_layout {
|
|||||||
check_like($unit, qr|--volume /srv/llm/modelscope:/root/\.cache/modelscope:Z|,
|
check_like($unit, qr|--volume /srv/llm/modelscope:/root/\.cache/modelscope:Z|,
|
||||||
'custom layout: the cache volume follows the state directory');
|
'custom layout: the cache volume follows the state directory');
|
||||||
unlink('/etc/systemd/system/llm.service');
|
unlink('/etc/systemd/system/llm.service');
|
||||||
unlink('/etc/nginx/conf.d/llm.conf');
|
unlink('/etc/caddy/Caddyfile.d/llm.caddyfile');
|
||||||
remove_tree('/etc/ssl/llm');
|
remove_tree('/etc/ssl/llm');
|
||||||
remove_tree('/srv/llm');
|
remove_tree('/srv/llm');
|
||||||
return;
|
return;
|
||||||
@@ -674,17 +819,32 @@ sub scenario_selinux {
|
|||||||
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set while permissive');
|
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set while permissive');
|
||||||
check_like($out, qr/SELinux: permissive \(skipped\)/, 'selinux: reported as skipped');
|
check_like($out, qr/SELinux: permissive \(skipped\)/, 'selinux: reported as skipped');
|
||||||
|
|
||||||
|
# Enforcing with no confined caddy policy: the distributions run caddy
|
||||||
|
# unconfined, so no boolean is touched.
|
||||||
reset_fixture();
|
reset_fixture();
|
||||||
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
write_fixture('rocm.txt', "7.2.4\n");
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
$ENV{STUB_SELINUX} = 'Enforcing';
|
$ENV{STUB_SELINUX} = 'Enforcing';
|
||||||
reset_log();
|
reset_log();
|
||||||
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
|
check(count_in_log(qr/^setsebool /) == 0, 'selinux: no boolean without a confined policy');
|
||||||
|
check_like($out, qr/SELinux: caddy runs unconfined \(nothing to do\)/,
|
||||||
|
'selinux: the unconfined case is stated');
|
||||||
|
|
||||||
|
# Enforcing with a confined caddy policy loaded: the boolean is set.
|
||||||
|
reset_fixture();
|
||||||
|
write_fixture('gpu.txt', "03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Instinct MI300X OAM [1002:74a1]\n");
|
||||||
|
write_fixture('rocm.txt', "7.2.4\n");
|
||||||
|
write_fixture('semodule-caddy', "1\n");
|
||||||
|
reset_log();
|
||||||
|
($rc, $out) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
|
check_like(stub_log(), qr/^semodule -l$/m, 'selinux: the loaded modules are asked for');
|
||||||
check_like(stub_log(), qr/^setsebool -P httpd_can_network_connect=1$/m, 'selinux: the boolean is set');
|
check_like(stub_log(), qr/^setsebool -P httpd_can_network_connect=1$/m, 'selinux: the boolean is set');
|
||||||
check_like($out, qr/SELinux: httpd_can_network_connect on/, 'selinux: reported as on');
|
check_like($out, qr/SELinux: httpd_can_network_connect on/, 'selinux: reported as on');
|
||||||
reset_log();
|
reset_log();
|
||||||
my ($rc2, $out2) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
my ($rc2, $out2) = run_script('--model', 'ZhipuAI/GLM-5.3');
|
||||||
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set when already on');
|
check(count_in_log(qr/^setsebool /) == 0, 'selinux: nothing set when already on');
|
||||||
|
unlink("$FIX/semodule-caddy");
|
||||||
delete $ENV{STUB_SELINUX};
|
delete $ENV{STUB_SELINUX};
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -695,6 +855,9 @@ my %scenarios = (
|
|||||||
dry_run => \&scenario_dry_run,
|
dry_run => \&scenario_dry_run,
|
||||||
uninstall => \&scenario_uninstall,
|
uninstall => \&scenario_uninstall,
|
||||||
uninstall_twice => \&scenario_uninstall_twice,
|
uninstall_twice => \&scenario_uninstall_twice,
|
||||||
|
legacy_nginx => \&scenario_legacy_nginx,
|
||||||
|
caddy_binary => \&scenario_caddy_binary,
|
||||||
|
epel => \&scenario_epel,
|
||||||
radeon => \&scenario_radeon,
|
radeon => \&scenario_radeon,
|
||||||
radeon_dev => \&scenario_radeon_dev,
|
radeon_dev => \&scenario_radeon_dev,
|
||||||
radeon_with_image => \&scenario_radeon_with_image,
|
radeon_with_image => \&scenario_radeon_with_image,
|
||||||
@@ -715,6 +878,7 @@ my %scenarios = (
|
|||||||
);
|
);
|
||||||
|
|
||||||
prepare_stubs();
|
prepare_stubs();
|
||||||
|
prepare_group();
|
||||||
prepare_devices();
|
prepare_devices();
|
||||||
|
|
||||||
my @wanted = @ARGV ? @ARGV : sort keys %scenarios;
|
my @wanted = @ARGV ? @ARGV : sort keys %scenarios;
|
||||||
|
|||||||
Regular → Executable
+71
-5
@@ -74,6 +74,12 @@ if ($name eq 'rpm') {
|
|||||||
|
|
||||||
if ($name eq 'dnf') {
|
if ($name eq 'dnf') {
|
||||||
my @pkgs = grep { !/^-/ && $_ ne 'install' } @args;
|
my @pkgs = grep { !/^-/ && $_ ne 'install' } @args;
|
||||||
|
# Parenthesised on purpose: a named list operator swallows a trailing &&,
|
||||||
|
# and the unparenthesised form asks the grep about a boolean, not the list.
|
||||||
|
if ((grep { $_ eq 'caddy' } @pkgs) && -f "$FIX/caddy-no-package") {
|
||||||
|
print STDERR "Error: Unable to find a match: caddy\n";
|
||||||
|
exit 1;
|
||||||
|
}
|
||||||
my $installed = fixture_text('rpm-installed', '');
|
my $installed = fixture_text('rpm-installed', '');
|
||||||
for my $pkg (@pkgs) {
|
for my $pkg (@pkgs) {
|
||||||
$installed .= "$pkg\n" unless $installed =~ /^\Q$pkg\E$/m;
|
$installed .= "$pkg\n" unless $installed =~ /^\Q$pkg\E$/m;
|
||||||
@@ -82,6 +88,12 @@ if ($name eq 'dnf') {
|
|||||||
print {$fh} $installed;
|
print {$fh} $installed;
|
||||||
close($fh);
|
close($fh);
|
||||||
}
|
}
|
||||||
|
# A package transaction that installs caddy leaves the binary where PATH
|
||||||
|
# finds it, the way the real package does. The grep is parenthesised as
|
||||||
|
# above: a named list operator swallows a trailing &&.
|
||||||
|
if ((grep { $_ eq 'caddy' } @pkgs) && !-e '/usr/bin/caddy') {
|
||||||
|
symlink($0, '/usr/bin/caddy');
|
||||||
|
}
|
||||||
print "Installing: @pkgs\n";
|
print "Installing: @pkgs\n";
|
||||||
exit 0;
|
exit 0;
|
||||||
}
|
}
|
||||||
@@ -138,14 +150,32 @@ if ($name eq 'curl') {
|
|||||||
print "\n__HTTP__$code\n" if $joined =~ /__HTTP__/;
|
print "\n__HTTP__$code\n" if $joined =~ /__HTTP__/;
|
||||||
exit 0;
|
exit 0;
|
||||||
}
|
}
|
||||||
|
if ($url =~ m{api\.github\.com/repos/caddyserver/caddy}) {
|
||||||
|
print fixture_text('caddy-release.json', qq({"tag_name":"v2.10.2"}\n));
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
if ($url =~ m{api\.github\.com}) {
|
if ($url =~ m{api\.github\.com}) {
|
||||||
print fixture_text('releases.json', qq({"tag_name":"v0.5.19"}\n));
|
print fixture_text('releases.json', qq({"tag_name":"v0.5.19"}\n));
|
||||||
exit 0;
|
exit 0;
|
||||||
}
|
}
|
||||||
|
if ($url =~ m{github\.com/caddyserver/caddy/releases/download/}) {
|
||||||
|
my $dest;
|
||||||
|
for my $i (0 .. $#args) {
|
||||||
|
$dest = $args[$i + 1] if $args[$i] eq '-o';
|
||||||
|
}
|
||||||
|
if (defined $dest) {
|
||||||
|
open(my $fh, '>', $dest) or exit 1;
|
||||||
|
print {$fh} "stub caddy release archive\n";
|
||||||
|
close($fh);
|
||||||
|
}
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
if ($url =~ m{hub\.docker\.com}) {
|
if ($url =~ m{hub\.docker\.com}) {
|
||||||
if (-f "$FIX/image-missing") {
|
# A definitive 404 on a tag lookup is what the script reads as
|
||||||
print STDERR "curl: (22) The requested URL returned error: 404\n";
|
# unpublished; a curl-level failure would read as cannot-tell.
|
||||||
exit 22;
|
if ($url =~ m{/tags/[A-Za-z0-9._-]+$} && -f "$FIX/image-missing") {
|
||||||
|
print "404";
|
||||||
|
exit 0;
|
||||||
}
|
}
|
||||||
if ($url =~ /tags\?/) {
|
if ($url =~ /tags\?/) {
|
||||||
# A tag listing, newest first: the rig's AMD development build.
|
# A tag listing, newest first: the rig's AMD development build.
|
||||||
@@ -181,8 +211,44 @@ if ($name eq 'openssl') {
|
|||||||
exit 0;
|
exit 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
if ($name eq 'nginx') {
|
if ($name eq 'caddy') {
|
||||||
print "nginx: configuration file /etc/nginx/nginx.conf test is successful\n";
|
my $joined = join(' ', @args);
|
||||||
|
if ($joined =~ /version/) {
|
||||||
|
print "v2.10.2 h1:stub\n";
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
|
if ($joined =~ /validate/) {
|
||||||
|
print "Valid configuration\n";
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($name eq 'tar') {
|
||||||
|
# The release-binary install extracts the archive and installs the binary it
|
||||||
|
# names; the stub lays down a link to this dispatcher, so the installed
|
||||||
|
# stand-in answers and logs like every other stubbed command.
|
||||||
|
my $dest_dir;
|
||||||
|
for my $i (0 .. $#args) {
|
||||||
|
$dest_dir = $args[$i + 1] if $args[$i] eq '-C';
|
||||||
|
}
|
||||||
|
if (defined $dest_dir) {
|
||||||
|
unlink("$dest_dir/caddy");
|
||||||
|
symlink($0, "$dest_dir/caddy");
|
||||||
|
}
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($name eq 'useradd') {
|
||||||
|
exit 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($name eq 'semodule') {
|
||||||
|
# A loaded module line looks like "100 caddy\tpp"; the fixture decides
|
||||||
|
# whether this host carries a confined caddy policy.
|
||||||
|
if (-f "$FIX/semodule-caddy") {
|
||||||
|
print "100 caddy\tpp\n";
|
||||||
|
}
|
||||||
exit 0;
|
exit 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -124,6 +124,25 @@ my %defaults = parse_args();
|
|||||||
is($defaults{dry_run}, 0, 'args: dry run is off by default');
|
is($defaults{dry_run}, 0, 'args: dry run is off by default');
|
||||||
@ARGV = @saved;
|
@ARGV = @saved;
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# The base package catalogue
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# The proxy of the collection is caddy everywhere; nginx left the baseline when
|
||||||
|
# sglang-deploy.pl moved to Caddy, and the name survives only in its own legacy
|
||||||
|
# clean-up.
|
||||||
|
check((grep { $_ eq 'caddy' } base_packages()) == 1, 'packages: caddy is a base package');
|
||||||
|
check((grep { $_ eq 'nginx' } base_packages()) == 0, 'packages: nginx is not');
|
||||||
|
is(join('|', base_packages()),
|
||||||
|
'nano|curl|wget|htop|tmux|rsync|caddy|openssl|jq|fastfetch',
|
||||||
|
'packages: the whole list, in order');
|
||||||
|
my %openeuler_gaps = unpackaged_for('openeuler');
|
||||||
|
is(exists $openeuler_gaps{caddy} ? 'yes' : 'no', 'yes',
|
||||||
|
'packages: the openEuler gap is named in the unpackaged map');
|
||||||
|
is(scalar(unpackaged_for('fedora')) // 0, 0, 'packages: Fedora has no unpackaged entry');
|
||||||
|
is(scalar(unpackaged_for('centos')) // 0, 0,
|
||||||
|
'packages: CentOS Stream has no unpackaged entry, EPEL carries caddy');
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# The command runner
|
# The command runner
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
+37
-16
@@ -138,7 +138,7 @@ is((valid_dir_path('/opt/sg lang') ? 1 : 0), 0, 'valid_dir_path: whitespace is r
|
|||||||
is((valid_dir_path('/opt/%h/sglang') ? 1 : 0), 0,
|
is((valid_dir_path('/opt/%h/sglang') ? 1 : 0), 0,
|
||||||
'valid_dir_path: a systemd specifier is refused');
|
'valid_dir_path: a systemd specifier is refused');
|
||||||
is((valid_dir_path('/opt/x;/sglang') ? 1 : 0), 0,
|
is((valid_dir_path('/opt/x;/sglang') ? 1 : 0), 0,
|
||||||
'valid_dir_path: an nginx directive end is refused');
|
'valid_dir_path: a path with a semicolon is refused');
|
||||||
|
|
||||||
# ---- run(): exit status, signals and timeout ------------------------------
|
# ---- run(): exit status, signals and timeout ------------------------------
|
||||||
my $killed = run([$^X, '-e', 'kill 9, $$']);
|
my $killed = run([$^X, '-e', 'kill 9, $$']);
|
||||||
@@ -164,22 +164,43 @@ is(scalar @{ radeon_env_for(undef, 1) }, 2,
|
|||||||
'env: a custom image on a Radeon-only host carries the Radeon defaults');
|
'env: a custom image on a Radeon-only host carries the Radeon defaults');
|
||||||
is(scalar @{ radeon_env_for('mi30x', 0) }, 0, 'env: an Instinct host carries none');
|
is(scalar @{ radeon_env_for('mi30x', 0) }, 0, 'env: an Instinct host carries none');
|
||||||
|
|
||||||
# ---- nginx config --------------------------------------------------------
|
# ---- caddy drop-in --------------------------------------------------------
|
||||||
my $conf = nginx_conf_content(8000, '[::1]', '/etc/ssl/sglang');
|
my $conf = caddyfile_content(8000, '[::1]', '/etc/ssl/sglang');
|
||||||
like($conf, qr/listen \[::\]:443 ssl;/, 'nginx: IPv6 listener on a dual-stack kernel');
|
like($conf, qr/^:443 \{$/m, 'caddy: the endpoint site on 443');
|
||||||
like($conf, qr/listen 443 ssl;/, 'nginx: IPv4 listener');
|
like($conf, qr|tls /etc/ssl/sglang/sglang\.crt /etc/ssl/sglang/sglang\.key|,
|
||||||
like($conf, qr|ssl_certificate /etc/ssl/sglang/sglang\.crt;|, 'nginx: certificate path');
|
'caddy: certificate pair paths');
|
||||||
like($conf, qr/ssl_certificate_key \/etc\/ssl\/sglang\/sglang\.key;/, 'nginx: key path');
|
like($conf, qr/reverse_proxy \[::1\]:8000 \{/, 'caddy: loopback upstream with the port');
|
||||||
like($conf, qr|proxy_pass http://\[::1\]:8000;|, 'nginx: loopback upstream with the port');
|
like($conf, qr/flush_interval -1/, 'caddy: unbuffered streaming');
|
||||||
like($conf, qr/proxy_http_version 1\.1;/, 'nginx: HTTP/1.1 for streaming');
|
like($conf, qr/header_up X-Real-IP \{remote_host\}/, 'caddy: X-Real-IP survives the heredoc');
|
||||||
like($conf, qr/proxy_buffering off;/, 'nginx: buffering off for streaming');
|
like($conf, qr/max_size 50MB/, 'caddy: the request body limit');
|
||||||
like($conf, qr/proxy_set_header Host \$host;/, 'nginx: $host survives the heredoc');
|
like($conf, qr/\treverse_proxy /, 'caddy: tab-indented as the Caddyfile is formatted');
|
||||||
like($conf, qr/proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;/,
|
check_unlike($conf, qr/\bbind\b/, 'caddy: no bind directive, the kernel decides the families');
|
||||||
'nginx: $proxy_add_x_forwarded_for survives the heredoc');
|
check_unlike($conf, qr/acme|on_demand|http:\/\/\{/i, 'caddy: no ACME, the self-signed pair is used');
|
||||||
|
|
||||||
my $conf4 = nginx_conf_content(8000, '127.0.0.1', '/etc/ssl/sglang');
|
my $conf4 = caddyfile_content(9000, '127.0.0.1', '/etc/ssl/llm');
|
||||||
is($conf4 =~ /\[::\]/ ? 'yes' : 'no',
|
like($conf4, qr/reverse_proxy 127\.0\.0\.1:9000 \{/, 'caddy: an IPv4 upstream carries the port');
|
||||||
(-e '/proc/net/if_inet6' ? 'yes' : 'no'), 'nginx: IPv6 listener follows the kernel');
|
like($conf4, qr|tls /etc/ssl/llm/sglang\.crt|, 'caddy: the certificate directory follows --cert-dir');
|
||||||
|
is(caddyfile_path('sglang'), '/etc/caddy/Caddyfile.d/sglang.caddyfile',
|
||||||
|
'caddy: the drop-in path follows the service name');
|
||||||
|
is(caddy_main_config(), '/etc/caddy/Caddyfile', 'caddy: the main Caddyfile path');
|
||||||
|
|
||||||
|
# ---- caddy release binary -------------------------------------------------
|
||||||
|
is(uname_to_arch('x86_64'), 'amd64', 'caddy binary: x86_64 maps to amd64');
|
||||||
|
is(uname_to_arch('aarch64'), 'arm64', 'caddy binary: aarch64 maps to arm64');
|
||||||
|
is(uname_to_arch('ppc64le'), undef, 'caddy binary: an unmapped machine is refused');
|
||||||
|
is(caddy_asset_url('2.10.2', 'amd64'),
|
||||||
|
'https://github.com/caddyserver/caddy/releases/download/v2.10.2/caddy_2.10.2_linux_amd64.tar.gz',
|
||||||
|
'caddy binary: the release asset URL');
|
||||||
|
|
||||||
|
my $caddy_unit = caddy_unit_content();
|
||||||
|
like($caddy_unit, qr|ExecStartPre=/usr/local/bin/caddy validate --config /etc/caddy/Caddyfile|,
|
||||||
|
'caddy binary: the unit validates before it starts');
|
||||||
|
like($caddy_unit, qr|ExecStart=/usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile|,
|
||||||
|
'caddy binary: the unit runs the release binary');
|
||||||
|
like($caddy_unit, qr|ExecReload=/usr/local/bin/caddy reload --config /etc/caddy/Caddyfile|,
|
||||||
|
'caddy binary: the unit reloads through the admin endpoint');
|
||||||
|
like($caddy_unit, qr/^User=caddy$/m, 'caddy binary: the unit runs as the caddy user');
|
||||||
|
like($caddy_unit, qr/AmbientCapabilities=CAP_NET_BIND_SERVICE/, 'caddy binary: the port capability');
|
||||||
|
|
||||||
# ---- systemd unit --------------------------------------------------------
|
# ---- systemd unit --------------------------------------------------------
|
||||||
my $unit = systemd_content({
|
my $unit = systemd_content({
|
||||||
|
|||||||
Reference in New Issue
Block a user