feat: full NFSv4.2 server and client in pure Go
Test / test (push) Successful in 2m4s
Release / gates (push) Successful in 2m5s
Release / build (amd64, freebsd) (push) Successful in 1m27s
Release / build (amd64, linux) (push) Successful in 1m22s
Release / build (amd64, netbsd) (push) Successful in 1m19s
Release / build (amd64, openbsd) (push) Successful in 1m20s
Release / build (arm64, darwin) (push) Successful in 1m21s
Release / build (arm64, freebsd) (push) Successful in 1m26s
Release / build (arm64, linux) (push) Successful in 1m25s
Release / build (arm64, netbsd) (push) Successful in 1m31s
Release / build (arm64, openbsd) (push) Successful in 1m27s
Release / build (loong64, linux) (push) Successful in 1m37s
Release / build (riscv64, linux) (push) Successful in 1m21s
Release / release (push) Successful in 40s
Test / test (push) Successful in 2m4s
Release / gates (push) Successful in 2m5s
Release / build (amd64, freebsd) (push) Successful in 1m27s
Release / build (amd64, linux) (push) Successful in 1m22s
Release / build (amd64, netbsd) (push) Successful in 1m19s
Release / build (amd64, openbsd) (push) Successful in 1m20s
Release / build (arm64, darwin) (push) Successful in 1m21s
Release / build (arm64, freebsd) (push) Successful in 1m26s
Release / build (arm64, linux) (push) Successful in 1m25s
Release / build (arm64, netbsd) (push) Successful in 1m31s
Release / build (arm64, openbsd) (push) Successful in 1m27s
Release / build (loong64, linux) (push) Successful in 1m37s
Release / build (riscv64, linux) (push) Successful in 1m21s
Release / release (push) Successful in 40s
Assisted-by: GLM 5.3 Flash
This commit is contained in:
@@ -0,0 +1,119 @@
|
||||
# Architecture
|
||||
|
||||
How nfs is put together. Every node, package and arrow below exists in the
|
||||
source tree; nothing is aspirational.
|
||||
|
||||
## Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
nfs[cmd/nfs] --> Client[internal/nfsclient]
|
||||
nfsd[cmd/nfsd] --> Server[internal/server]
|
||||
nfsd --> Backend[internal/nfsfs]
|
||||
Server --> Dispatch[internal/nfs4server]
|
||||
Dispatch --> Backend
|
||||
Dispatch --> Wire[internal/nfs4]
|
||||
Dispatch --> Record[internal/rpc]
|
||||
Client --> Wire
|
||||
Client --> Record
|
||||
Wire --> XDR[internal/xdr]
|
||||
Record --> XDR
|
||||
```
|
||||
|
||||
The project speaks NFSv4.2 only, on one TCP port, with the full state model
|
||||
of RFC 8881 under the extensions of RFC 7862 and the add-ons of RFC 8276,
|
||||
RFC 7861 and RFC 9289. The server carries the stateless operations, the
|
||||
session machinery, open and lock state, delegations with their back channel
|
||||
recalls, the optional operations, extended and named attributes, directory
|
||||
delegations with notifications, pNFS in the metadata server role, and the
|
||||
security layers: RPCSEC_GSS v1 and v3 with Kerberos in pure Go, and
|
||||
RPC-with-TLS with in place connection upgrade. The client mirrors the same
|
||||
surface and serves as the second oracle against the server. The two
|
||||
commands carry the roles: nfsd serves one local directory tree, nfs speaks
|
||||
to a server from the command line.
|
||||
|
||||
## Packages
|
||||
|
||||
| Package | Responsibility |
|
||||
|---|---|
|
||||
| `cmd/nfsd` | flags, the version report, the signal wiring; no logic |
|
||||
| `cmd/nfs` | the client command: the argument handling and the compound building for ls, cat, put and stat |
|
||||
| `internal/server` | the accept loop, connection lifetime and shutdown |
|
||||
| `internal/nfs4server` | the COMPOUND dispatcher: the file handle register, the session machinery, the operations, the mapping of backend errors to statuses |
|
||||
| `internal/nfs4` | the NFSv4.2 wire vocabulary: numbers, bitmap4, fattr4, the COMPOUND codec and the per operation arguments and results |
|
||||
| `internal/nfsfs` | the virtual filesystem interface and the local backend with dev and ino based handles |
|
||||
| `internal/rpc` | ONC RPC: record marking, the call and reply headers, AUTH_SYS credentials |
|
||||
| `internal/xdr` | the RFC 4506 primitives: integers, booleans, opaque values and strings with their padding |
|
||||
| `internal/nfsclient` | the client half; it shares the wire packages with the server and serves as the second oracle |
|
||||
| `internal/krb5` | the Kerberos crypto profiles and GSS tokens of RFC 3961, 3962, 4120 and 4121, verified against the test vectors of the RFCs and the MIT krb5 suite |
|
||||
| `internal/rdma` | the RPC-over-RDMA framing of RFC 8166: the fixed header, the chunk lists and the stream adapter; verbs live outside pure Go |
|
||||
|
||||
The boundaries follow the layering of the protocol stack: a layer speaks
|
||||
only to the one below it, and the wire packages know nothing about sockets.
|
||||
The dispatcher decides nothing about storage; the backend interface owns
|
||||
that.
|
||||
|
||||
## Data flow
|
||||
|
||||
The main operation today is one COMPOUND through the whole stack, from the
|
||||
client that is also the project's oracle:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as internal/nfsclient
|
||||
participant S as nfsd
|
||||
participant D as internal/nfs4server
|
||||
participant F as internal/nfsfs
|
||||
C->>S: record: COMPOUND, PUTROOTFH, LOOKUP, GETFH, GETATTR, READ
|
||||
S->>D: the record is reassembled, the call decoded
|
||||
D->>F: Root, Lookup, Getattr, Read
|
||||
F-->>D: handle, attributes, bytes
|
||||
D-->>C: one result per operation, the failing one ends the array
|
||||
```
|
||||
|
||||
READDIR carries the paging machinery of the standard: the client's cookie
|
||||
and verifier are checked against the backend's order, the entries are
|
||||
packed under the maxcount budget, and the response reports where the next
|
||||
page starts. The error paths are named: a clean end of stream before the
|
||||
first header is io.EOF, a stream that stops mid record is
|
||||
io.ErrUnexpectedEOF, a record beyond the limit is ErrRecordTooLarge, and
|
||||
every backend failure becomes the NFS4ERR status its sentinel names.
|
||||
|
||||
## State and lifetime
|
||||
|
||||
- The Server lives for the process and Serve blocks for that long; a
|
||||
cancellation of the context closes the listener and Serve returns nil.
|
||||
- Each accepted connection runs on its own goroutine and is owned by its
|
||||
Handle.
|
||||
- The file handle register is per COMPOUND: PUTROOTFH, PUTFH, SAVEFH and
|
||||
RESTOREFH move handles through it, and nothing of it survives the call.
|
||||
- Session state lives in the session store: the slot table with the reply
|
||||
cache, the lease of every client, the open and lock state with their
|
||||
stateids, delegations, directory delegations and the GSS contexts of
|
||||
RPCSEC_GSS. Referral stubs and named attribute handles are synthetic and
|
||||
live in their own stores.
|
||||
- The handles of the local backend encode the device and inode number and
|
||||
resolve through an in memory map persisted on demand: a handle from
|
||||
before a server restart resolves again once the mapping is loaded back,
|
||||
and one whose object is gone answers NFS4ERR_STALE.
|
||||
- WRITE answers FILE_SYNC with the boot verifier, so no unstable writes
|
||||
outlive a restart and the client keeps no replay debt. The stateid of a
|
||||
WRITE is validated: a real one must name a live OPEN of the current file,
|
||||
and the anonymous forms pass without state.
|
||||
- WRITE and CREATE reach the backend through the nfsfs Writer interface; a
|
||||
backend that does not implement it is answered NFS4ERR_ROFS. The local
|
||||
backend implements both halves: CREATE makes directories, symlinks,
|
||||
fifos, sockets and device nodes, and a regular file is the business of
|
||||
OPEN.
|
||||
|
||||
## Dependencies
|
||||
|
||||
One dependency outside the standard library: interpres
|
||||
(`sourcedock.dev/petrbalvin/interpres/v2`), the TOML reader of the
|
||||
configuration file, itself built on the standard library alone. The
|
||||
protocol stack is carried by hand written code, because the server
|
||||
targets Linux on amd64, arm64, loong64 and riscv64; FreeBSD,
|
||||
OpenBSD and NetBSD on amd64 and arm64; and darwin on arm64,
|
||||
and a dependency
|
||||
that breaks one of those platforms is a dependency the project cannot
|
||||
carry.
|
||||
Reference in New Issue
Block a user