petrbalvin/scripts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: describe the sglang endpoint behind caddy

Browse Source 
Assisted-by: GLM 5.3 Flash
This commit is contained in:
petrbalvin
2026-09-29 00:18:15 +02:00
parent 9455c65f10
commit 37e06d090c
6 changed files with 35 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+15 -3
View File
@@ -37,7 +37,7 @@ reports everything as already in place and changes nothing.
| `system-diag.pl` | Reading CPU, memory, disk, network, GPU, services, security and performance, and grading the result | Change anything on the host; it is read-only |
| `server-setup.pl` | Base packages, firewall (with the SSH rule verified before the service starts), SELinux, Podman, automatic updates | Install the services themselves; that is the operator's, and the deploy scripts' |
| `workstation-setup.pl` | A Fedora desktop: Brave, the official Go toolchain, Rust, GoLand, Flatpak applications, firewall, SELinux | Anything on a server distribution; it targets Fedora Workstation |
| `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, nginx with TLS, the API key, the firewall rule and the SELinux boolean | The host's own setup, which `server-setup.pl` does first |
| `sglang-deploy.pl` | The SGLang deployment: the container engine, the systemd unit, Caddy with TLS, the API key, the firewall rule and the SELinux checks | The host's own setup, which `server-setup.pl` does first |
| `system-optimise.pl` | Old kernels (the running one and one fallback always stay), journals, temporary files, core dumps, and the package audit | Touch an rpm-ostree system, which it refuses |
## Data flow: a forked section collection
@@ -97,8 +97,8 @@ that CentOS Stream and openEuler cannot carry at all.
```mermaid
flowchart LR
Client[client] -->|443| Nginx[nginx on the host, TLS]
Nginx -->|loopback, plain HTTP| Engine[SGLang in a Podman container]
Client[client] -->|443| Caddy[Caddy on the host, TLS]
Caddy -->|loopback, plain HTTP| Engine[SGLang in a Podman container]
Engine -->|device nodes| GPU[/dev/kfd, /dev/dri]
Engine -->|bind mount| Cache[state directory, model cache]
Unit[systemd unit] -->|podman run| Engine
@@ -114,12 +114,24 @@ instead, and `--image` pins one. Radeon cards need `SGLANG_USE_AITER=false` and
`SGLANG_ROCM_FUSED_DECODE_MLA=false` in the unit, which the script writes for them and
never for an Instinct host.
Caddy serves the endpoint from a drop-in under `/etc/caddy/Caddyfile.d`, which the
distribution's default Caddyfile imports. Fedora carries the caddy package and CentOS
Stream gets it from EPEL, whose repository file the script installs first; openEuler
packages no caddy at all, so there the official release binary is installed instead,
with the unit file the package would have carried. The service runs as the caddy user,
so the private key is made group-readable for the caddy group, and on an
SELinux-enforcing host the packaged caddy runs unconfined: no boolean is needed. A
deployment made by an earlier release of this script carries an nginx configuration,
which both a deploy and an uninstall remove.
## Dependencies
Nothing outside the interpreter, and nothing that has to be installed beyond the tools
each script's own dependency section installs. The non-obvious ones and their reasons:
- `podman` for `sglang-deploy.pl`, because the engine is a container.
- `caddy` for `sglang-deploy.pl`, because the endpoint serves TLS on 443; the
release binary and `tar` stand in where no repository carries the package.
- `lspci` for the GPU family, and `rocm-smi` in `system-diag.pl` for AMD memory and
utilisation figures.
- `sha256sum` in `workstation-setup.pl`, because Perl's builtins have no hash.