nfs/docs/ARCHITECTURE.md

# Architecture

How nfs is put together. Every node, package and arrow below exists in the
source tree; nothing is aspirational.

## Overview

```mermaid
flowchart TD
    nfs[cmd/nfs] --> Client[internal/nfsclient]
    nfsd[cmd/nfsd] --> Server[internal/server]
    nfsd --> Backend[internal/nfsfs]
    Server --> Dispatch[internal/nfs4server]
    Dispatch --> Backend
    Dispatch --> Wire[internal/nfs4]
    Dispatch --> Record[internal/rpc]
    Client --> Wire
    Client --> Record
    Wire --> XDR[internal/xdr]
    Record --> XDR
```

The project speaks NFSv4.2 only, on one TCP port, with the full state model
of RFC 8881 under the extensions of RFC 7862 and the add-ons of RFC 8276,
RFC 7861 and RFC 9289. The server carries the stateless operations, the
session machinery, open and lock state, delegations with their back channel
recalls, the optional operations, extended and named attributes, directory
delegations with notifications, pNFS in the metadata server role, and the
security layers: RPCSEC_GSS v1 and v3 with Kerberos in pure Go, and
RPC-with-TLS with in place connection upgrade. The client mirrors the same
surface and serves as the second oracle against the server. The two
commands carry the roles: nfsd serves one local directory tree, nfs speaks
to a server from the command line.

## Packages

| Package | Responsibility |
|---|---|
| `cmd/nfsd` | flags, the version report, the signal wiring; no logic |
| `cmd/nfs` | the client command: the argument handling and the compound building for ls, cat, put and stat |
| `internal/server` | the accept loop, connection lifetime and shutdown |
| `internal/nfs4server` | the COMPOUND dispatcher: the file handle register, the session machinery, the operations, the mapping of backend errors to statuses |
| `internal/nfs4` | the NFSv4.2 wire vocabulary: numbers, bitmap4, fattr4, the COMPOUND codec and the per operation arguments and results |
| `internal/nfsfs` | the virtual filesystem interface and the local backend with dev and ino based handles |
| `internal/rpc` | ONC RPC: record marking, the call and reply headers, AUTH_SYS credentials |
| `internal/xdr` | the RFC 4506 primitives: integers, booleans, opaque values and strings with their padding |
| `internal/nfsclient` | the client half; it shares the wire packages with the server and serves as the second oracle |
| `internal/krb5` | the Kerberos crypto profiles and GSS tokens of RFC 3961, 3962, 4120 and 4121, verified against the test vectors of the RFCs and the MIT krb5 suite |
| `internal/rdma` | the RPC-over-RDMA framing of RFC 8166: the fixed header, the chunk lists and the stream adapter; verbs live outside pure Go |

The boundaries follow the layering of the protocol stack: a layer speaks
only to the one below it, and the wire packages know nothing about sockets.
The dispatcher decides nothing about storage; the backend interface owns
that.

## Data flow

The main operation today is one COMPOUND through the whole stack, from the
client that is also the project's oracle:

```mermaid
sequenceDiagram
    participant C as internal/nfsclient
    participant S as nfsd
    participant D as internal/nfs4server
    participant F as internal/nfsfs
    C->>S: record: COMPOUND, PUTROOTFH, LOOKUP, GETFH, GETATTR, READ
    S->>D: the record is reassembled, the call decoded
    D->>F: Root, Lookup, Getattr, Read
    F-->>D: handle, attributes, bytes
    D-->>C: one result per operation, the failing one ends the array
```

READDIR carries the paging machinery of the standard: the client's cookie
and verifier are checked against the backend's order, the entries are
packed under the maxcount budget, and the response reports where the next
page starts. The error paths are named: a clean end of stream before the
first header is io.EOF, a stream that stops mid record is
io.ErrUnexpectedEOF, a record beyond the limit is ErrRecordTooLarge, and
every backend failure becomes the NFS4ERR status its sentinel names.

## State and lifetime

- The Server lives for the process and Serve blocks for that long; a
  cancellation of the context closes the listener and Serve returns nil.
- Each accepted connection runs on its own goroutine and is owned by its
  Handle.
- The file handle register is per COMPOUND: PUTROOTFH, PUTFH, SAVEFH and
  RESTOREFH move handles through it, and nothing of it survives the call.
- Session state lives in the session store: the slot table with the reply
  cache, the lease of every client, the open and lock state with their
  stateids, delegations, directory delegations and the GSS contexts of
  RPCSEC_GSS. Referral stubs and named attribute handles are synthetic and
  live in their own stores.
- The handles of the local backend encode the device and inode number and
  resolve through an in memory map persisted on demand: a handle from
  before a server restart resolves again once the mapping is loaded back,
  and one whose object is gone answers NFS4ERR_STALE.
- WRITE answers FILE_SYNC with the boot verifier, so no unstable writes
  outlive a restart and the client keeps no replay debt. The stateid of a
  WRITE is validated: a real one must name a live OPEN of the current file,
  and the anonymous forms pass without state.
- WRITE and CREATE reach the backend through the nfsfs Writer interface; a
  backend that does not implement it is answered NFS4ERR_ROFS. The local
  backend implements both halves: CREATE makes directories, symlinks,
  fifos, sockets and device nodes, and a regular file is the business of
  OPEN.

## Dependencies

One dependency outside the standard library: interpres
(`sourcedock.dev/petrbalvin/interpres/v2`), the TOML reader of the
configuration file, itself built on the standard library alone. The
protocol stack is carried by hand written code, because the server
targets Linux on amd64, arm64, loong64 and riscv64; FreeBSD,
OpenBSD and NetBSD on amd64 and arm64; and darwin on arm64,
and a dependency
that breaks one of those platforms is a dependency the project cannot
carry.