nfs/docs/ARCHITECTURE.md

Architecture

How nfs is put together. Every node, package and arrow below exists in the source tree; nothing is aspirational.

Overview

flowchart TD
    nfs[cmd/nfs] --> Client[internal/nfsclient]
    nfsd[cmd/nfsd] --> Server[internal/server]
    nfsd --> Backend[internal/nfsfs]
    Server --> Dispatch[internal/nfs4server]
    Dispatch --> Backend
    Dispatch --> Wire[internal/nfs4]
    Dispatch --> Record[internal/rpc]
    Client --> Wire
    Client --> Record
    Wire --> XDR[internal/xdr]
    Record --> XDR

The project speaks NFSv4.2 only, on one TCP port, with the full state model of RFC 8881 under the extensions of RFC 7862 and the add-ons of RFC 8276, RFC 7861 and RFC 9289. The server carries the stateless operations, the session machinery, open and lock state, delegations with their back channel recalls, the optional operations, extended and named attributes, directory delegations with notifications, pNFS in the metadata server role, and the security layers: RPCSEC_GSS v1 and v3 with Kerberos in pure Go, and RPC-with-TLS with in place connection upgrade. The client mirrors the same surface and serves as the second oracle against the server. The two commands carry the roles: nfsd serves one local directory tree, nfs speaks to a server from the command line.

Packages

Package	Responsibility
cmd/nfsd	flags, the version report, the signal wiring; no logic
cmd/nfs	the client command: the argument handling and the compound building for ls, cat, put and stat
internal/server	the accept loop, connection lifetime and shutdown
internal/nfs4server	the COMPOUND dispatcher: the file handle register, the session machinery, the operations, the mapping of backend errors to statuses
internal/nfs4	the NFSv4.2 wire vocabulary: numbers, bitmap4, fattr4, the COMPOUND codec and the per operation arguments and results
internal/nfsfs	the virtual filesystem interface and the local backend with dev and ino based handles
internal/rpc	ONC RPC: record marking, the call and reply headers, AUTH_SYS credentials
internal/xdr	the RFC 4506 primitives: integers, booleans, opaque values and strings with their padding
internal/nfsclient	the client half; it shares the wire packages with the server and serves as the second oracle
internal/krb5	the Kerberos crypto profiles and GSS tokens of RFC 3961, 3962, 4120 and 4121, verified against the test vectors of the RFCs and the MIT krb5 suite
internal/rdma	the RPC-over-RDMA framing of RFC 8166: the fixed header, the chunk lists and the stream adapter; verbs live outside pure Go

The boundaries follow the layering of the protocol stack: a layer speaks only to the one below it, and the wire packages know nothing about sockets. The dispatcher decides nothing about storage; the backend interface owns that.

Data flow

The main operation today is one COMPOUND through the whole stack, from the client that is also the project's oracle:

sequenceDiagram
    participant C as internal/nfsclient
    participant S as nfsd
    participant D as internal/nfs4server
    participant F as internal/nfsfs
    C->>S: record: COMPOUND, PUTROOTFH, LOOKUP, GETFH, GETATTR, READ
    S->>D: the record is reassembled, the call decoded
    D->>F: Root, Lookup, Getattr, Read
    F-->>D: handle, attributes, bytes
    D-->>C: one result per operation, the failing one ends the array

READDIR carries the paging machinery of the standard: the client's cookie and verifier are checked against the backend's order, the entries are packed under the maxcount budget, and the response reports where the next page starts. The error paths are named: a clean end of stream before the first header is io.EOF, a stream that stops mid record is io.ErrUnexpectedEOF, a record beyond the limit is ErrRecordTooLarge, and every backend failure becomes the NFS4ERR status its sentinel names.

State and lifetime

• The Server lives for the process and Serve blocks for that long; a cancellation of the context closes the listener and Serve returns nil.
• Each accepted connection runs on its own goroutine and is owned by its Handle.
• The file handle register is per COMPOUND: PUTROOTFH, PUTFH, SAVEFH and RESTOREFH move handles through it, and nothing of it survives the call.
• Session state lives in the session store: the slot table with the reply cache, the lease of every client, the open and lock state with their stateids, delegations, directory delegations and the GSS contexts of RPCSEC_GSS. Referral stubs and named attribute handles are synthetic and live in their own stores.
• The handles of the local backend encode the device and inode number and resolve through an in memory map persisted on demand: a handle from before a server restart resolves again once the mapping is loaded back, and one whose object is gone answers NFS4ERR_STALE.
• WRITE answers FILE_SYNC with the boot verifier, so no unstable writes outlive a restart and the client keeps no replay debt. The stateid of a WRITE is validated: a real one must name a live OPEN of the current file, and the anonymous forms pass without state.
• WRITE and CREATE reach the backend through the nfsfs Writer interface; a backend that does not implement it is answered NFS4ERR_ROFS. The local backend implements both halves: CREATE makes directories, symlinks, fifos, sockets and device nodes, and a regular file is the business of OPEN.

Dependencies

One dependency outside the standard library: interpres (sourcedock.dev/petrbalvin/interpres/v2), the TOML reader of the configuration file, itself built on the standard library alone. The protocol stack is carried by hand written code, because the server targets Linux on amd64, arm64, loong64 and riscv64; FreeBSD, OpenBSD and NetBSD on amd64 and arm64; and darwin on arm64, and a dependency that breaks one of those platforms is a dependency the project cannot carry.