# Contributing Thank you for considering contributing to **goget** — a modern IPv6-first download utility built on the Go standard library. ## Development Setup ### Requirements - Go 1.26+ - Linux (amd64, arm64, riscv64, loong64) or FreeBSD (amd64, arm64, riscv64) ### Quick Start ```bash git clone https://codeberg.org/petrbalvin/goget.git cd goget just install # go mod download just test # go test -race -count=1 ./... just build # go vet + gofmt check + build → bin/goget ``` ### Running Tests ```bash just test ``` The test suite (`*_test.go` files alongside the code) covers: - Protocol handlers (HTTP, FTP, SFTP, WebDAV, file, data, Gopher, Gemini). - Transport layer (IPv6-first dialer, proxy, retry, rate limiting). - Checksum verification, PGP signature handling, HSTS cache. - Recursive downloading, link rewriting, Metalink parsing. - Configuration loading, merging, and sanitization. - CLI flag parsing, output formatting, progress bar. ### Linting ```bash just build # go vet + gofmt check (also part of CI) ``` CI runs both `go vet ./...` and a `gofmt -l` diff check. These are enforced — make sure they pass before submitting a pull request. ## Code Style - Follow [Effective Go](https://go.dev/doc/effective_go) conventions. - Keep lines under 100 characters where practical. - Use `camelCase` for variables and functions, `PascalCase` for exported symbols. - Prefix every file with a one-line comment describing its purpose. - Use build tags `//go:build linux || freebsd` on all source files. - Use long flags in CLI (`--header` not `-H`). - Prefer the Go standard library over external dependencies. External dependencies are only allowed from `golang.org/x/` unless justified. - Error messages in English, lowercase, no trailing period, always with context (`open config: permission denied`). - `gofmt` all code before committing. ### Testing - Write tests alongside the code in `_test.go` files. - Use table-driven tests for multiple test cases. - Include both happy path and error path tests. - Name tests following Go convention: `TestFunctionName_Scenario`. - Use `testing.F` for fuzz tests in security-critical parsing code. ## Commit Conventions We follow [Conventional Commits](https://www.conventionalcommits.org/). | Type | Use | |------------|----------------------------------------------| | `feat` | New feature | | `fix` | Bug fix | | `refactor` | Restructuring without behaviour change | | `docs` | Documentation | | `test` | Adding or updating tests | | `chore` | Maintenance, CI, tooling | | `perf` | Performance improvement | ``` feat: add Gemini protocol support fix: handle timeout during parallel chunk downloads refactor: unify transport dialer configuration docs: add README quick start examples test: add edge cases for cron schedule parser chore: update golang.org/x/crypto ``` - English, lowercase subject, imperative mood (`add`, not `added`). - No trailing period, max 72 characters. - No version numbers in commit messages — versions belong to tags. ## Pull Request Flow 1. Create a feature branch from `development`. 2. Make your changes with Conventional-Commits messages. 3. Record your changes under the `## [development]` section at the top of `CHANGELOG.md`, using the Keep a Changelog categories (`Added`, `Changed`, `Fixed`, etc.). 4. Ensure `just build` and `just test` pass. 5. Add or update tests for your change. 6. Update documentation if the public API, configuration, or behaviour changes: - `README.md` — high-level overview. - `docs/configuration.md` — config keys. - `docs/api.md` — Go library API. - `docs/cli-reference.md` — CLI flags. - `docs/architecture.md` — structural changes. - `docs/security.md` — anything touching auth, TLS, or verification. 7. Open a pull request against `development`. Releases are cut by merging `development` into `main` and pushing a `vX.Y.Z` tag — see [Cutting a release](#cutting-a-release). ## Project Structure ``` goget/ ├── cmd/ │ └── goget/ # CLI entry point │ ├── main.go # Core run loop, flag setup, orchestrator │ ├── app.go # Application wiring │ ├── commands.go # Command handlers (download, upload, queue, config, etc.) │ ├── flags.go # CLI flag definitions (struct + defineFlags) │ ├── completion.go # Shell completion generators (bash, zsh, fish) │ ├── man_page.go # Embedded manpage source and --generate-man-page │ ├── setup.go # One-time CLI setup helpers │ ├── schedule.go # Cron and datetime scheduling │ └── verify.go # Checksum and PGP verification ├── internal/ │ ├── archive/ # Archive extraction (tar, targz, tarbz2, zip) │ ├── auth/ # Authentication (basic, digest, OAuth 2.0, netrc) │ ├── cli/ # CLI utilities (help text, progress bar, color output) │ ├── compression/ # Decompression (gzip, bzip2, zlib, flate, lzw) │ ├── config/ # TOML config loading, saving, and merging │ ├── cookie/ # Netscape-format cookie jar │ ├── core/ # Core types (DownloadRequest, Result, Protocol interface, errors) │ ├── crypto/ # Checksum verification, PGP, certificate handling │ ├── format/ # Human-readable formatting (bytes, speed) │ ├── hsts/ # HSTS cache (RFC 6797) │ ├── linkrewrite/ # HTML link rewriting for offline mirroring │ ├── log/ # Structured logger (text/JSON output, level filtering) │ ├── metalink/ # Metalink (multi-source downloads) │ ├── mirror/ # Website mirroring logic │ ├── output/ # Output writing (atomic writes, resume metadata) │ ├── pool/ # Unified recursive worker pool (FTP, SFTP, WebDAV) │ ├── protocol/ # Protocol abstraction layer │ │ ├── http/ # HTTP/HTTPS client, parallel download, upload │ │ ├── ftp/ # FTP/FTPS client │ │ ├── sftp/ # SFTP client │ │ ├── file/ # Local file:// protocol │ │ ├── dataurl/ # data: URI protocol │ │ ├── gopher/ # Gopher protocol │ │ ├── gemini/ # Gemini protocol │ │ ├── webdav/ # WebDAV protocol │ │ └── register/ # Single registration entry point │ ├── queue/ # Download queue management │ ├── recursive/ # Recursive download (HTML parser, URL filter, crawler) │ ├── transport/ # HTTP transport (dialer, proxy, TLS, retry, rate limiting) │ └── warc/ # WARC archiving writer ├── pkg/ │ └── api/ # Public library API (Client, Download, Upload, Queue) └── docs/ # Architecture and design documentation ``` ## Architecture Overview ```mermaid graph TD CLI[cmd/goget/ - CLI Entry] Flags[Flags parsing] Config[internal/config/] Core[internal/core/ - Types & Protocol Interface] Registry[internal/protocol/ - Registry] HTTP[internal/protocol/http/] FTP[internal/protocol/ftp/] SFTP[internal/protocol/sftp/] FILE[internal/protocol/file/] DataURL[internal/protocol/dataurl/] Gopher[internal/protocol/gopher/] Gemini[internal/protocol/gemini/] WebDAV[internal/protocol/webdav/] Transport[internal/transport/ - Dialer, Proxy, TLS] Output[internal/output/ - Atomic write, Resume] Crypto[internal/crypto/ - Checksum, PGP] Auth[internal/auth/] Cookie[internal/cookie/] HSTS[internal/hsts/] Recursive[internal/recursive/] Mirror[internal/mirror/] Metalink[internal/metalink/] QueueInt[internal/queue/] Archive[internal/archive/] Pool[internal/pool/ - Unified worker pool] Log[internal/log/ - Structured logger] API[pkg/api/ - Public Library] CLI --> Flags CLI --> Config CLI --> Core CLI --> Log CLI --> Auth CLI --> Crypto CLI --> Recursive CLI --> Mirror CLI --> Metalink CLI --> QueueInt CLI --> Archive Core --> Registry Registry --> HTTP Registry --> FTP Registry --> SFTP Registry --> FILE Registry --> DataURL Registry --> Gopher Registry --> Gemini Registry --> WebDAV HTTP --> Transport HTTP --> Cookie HTTP --> HSTS HTTP --> Output FTP --> Pool SFTP --> Pool WebDAV --> Pool API --> Core API --> Registry ``` ### Key Design Decisions - **Protocol abstraction** — Every network protocol implements `core.Protocol` and registers with a global registry. The CLI and API resolve protocols by URL scheme at runtime. - **IPv6-first** — The transport dialer prefers IPv6 addresses and falls back to IPv4 only when no IPv6 address is available. - **Parallel downloads** — Files over 100 MB are automatically split into chunks. Each chunk downloads via a separate connection with byte-range requests. - **Atomic output** — Downloads write to a temporary file first, then atomically rename on completion. Resume metadata is stored alongside the partial file. - **Config merging** — CLI flags override config file values, which override defaults. The `config.Merge()` method applies CLI overrides after loading. - **No global state** — The CLI is the only singleton. Internal packages are designed for testability with injectable dependencies. ## Testing Strategy | Layer | Approach | Tools | |----------------|------------------------------------------------------------|---------------------| | Unit tests | Table-driven tests for parsers, validators, handlers | `testing` stdlib | | Integration | HTTP/HTTPS against local test servers, FTP against mock | `net/http/httptest` | | Fuzz tests | Random input for checksum parsing, Metalink, HSTS cache | `testing.F` | | Race detection | All tests run with `-race` in CI | `go test -race` | ## CI goget uses **Forgejo Actions**; both workflows live under `.forgejo/workflows/` and run on the `codeberg-small` / `codeberg-medium` runners. ### Test — `.forgejo/workflows/test.yml` Triggered on push and pull requests targeting `development`. | Job | What it does | |-------|-----------------------------------------------| | `vet` | `go vet ./...` (must report zero warnings) | | `test`| `go test -race -count=1 ./...` + coverage gate | | `fuzz`| `go test -fuzz` on every `FuzzXxx` function | Run the same locally before opening a PR: ```bash go vet ./... just test ``` ### Release — `.forgejo/workflows/release.yml` Triggered by pushing a `v*` tag. It verifies the tag matches `internal/core.Version`, runs the quality gate (go vet + tests), builds static binaries for all 7 platform targets, extracts the matching `CHANGELOG.md` section, creates a Forgejo release via the REST API, and attaches the binaries. Requires a `FORGEJO_TOKEN` secret with permission to create releases. ### Cutting a release 1. Bump the `Version` var in `internal/core/version.go`. 2. Rename `## [development]` to `## [X.Y.Z] — YYYY-MM-DD` at the top of `CHANGELOG.md`, and add a fresh empty `## [development]` section above it. 3. Ensure CI is green on `development`. 4. Merge `development` into `main` (the only time `main` changes outside a tag). 5. Tag and push: ```bash git tag -a vX.Y.Z -m "goget vX.Y.Z" git push origin main git push origin vX.Y.Z ``` 6. The release workflow builds the binaries and publishes the release from the CHANGELOG section. ## Report a Bug Open an issue at with: - Goget version (`goget --version`). - Operating system and architecture. - Command used and full output. - Expected vs actual behaviour. For **security issues**, please email **opensource@petrbalvin.org** rather than opening a public issue.