feat: initial goget release — modern IPv6-first download utility

2026-06-26 21:21:58 +02:00
commit 53db81e1f7
147 changed files with 45931 additions and 0 deletions
@@ -0,0 +1,187 @@
+# Backlog — goget
+
+This file tracks planned features and improvements across releases.
+Items are grouped by target version and priority. Completed items
+move to `CHANGELOG.md` when the version ships.
+
+---
+
+## v1.1.0 — Protocol completeness
+
+Finish the protocol layer so every supported protocol handles
+recursive, parallel, resume, and shutdown correctly.
+
+### 🔴 Supreme Priority
+
+- [ ] **Graceful shutdown for all protocols** — persist `.goget.meta` on
+  SIGINT for FTP single-file and SFTP parallel recursive paths.
+- [ ] **WebDAV connection pooling** — shared `http.Transport` across WebDAV
+  requests instead of a fresh transport per request (currently defeats HTTP
+  keep-alive, TLS session resumption, and HTTP/2 multiplexing).
+
+### 🟡 High Priority
+
+- [ ] **WebDAV recursive parallel downloads** — extend `--recursive-parallel`
+  to WebDAV using the shared transport.
+- [ ] **SFTP recursive parallel downloads** — parallel SFTP recursive with
+  session pooling.
+- [ ] **FTP recursive parallel** — connection pool for independent control
+  connections in FTP recursive mode.
+- [ ] **`--dry-run` for WebDAV recursive** — list matched entries without
+  downloading.
+
+### 🟢 Medium Priority
+
+- [ ] **`--adjust-extension` for non-HTML protocols** — apply `.html`
+  extension to text/html files from WebDAV, Gopher.
+- [ ] **`--backup-converted` for WebDAV mirrors** — keep `.orig` backups
+  before link conversion.
+- [ ] **Parallel recursive across all protocols** — unified worker pool
+  independent of protocol handler.
+
+---
+
+## v1.2.0 — curl/wget parity & UX
+
+Close the gap with curl and wget, and improve scripting/automation UX.
+
+### 🟡 High Priority
+
+- [ ] **`--create-dirs`** — auto-create parent directories for `--output`
+  paths (curl `--create-dirs` / wget `--directory-prefix` equivalent).
+- [ ] **Short flags** — `-O`, `-o`, `-s`, `-v` aliases for curl/wget
+  muscle memory. `--output-document` alias for `--output`.
+- [ ] **Brotli decompression** — RFC 7932, written from scratch.
+  Currently only gzip/deflate/bzip2/LZW.
+
+### 🟢 Medium Priority
+
+- [ ] **`--progress=bar:force:noscroll`** — curl-compatible progress options.
+- [ ] **`--metalink --follow-metalink=mem`** — keep Metalink in memory instead
+  of writing to disk.
+- [ ] **Bandwidth throttling per-domain** — global rate limit only at present.
+- [ ] **MIME type detection for local files** — determine Content-Type from
+  file extension when uploading.
+- [ ] **`--cut-dirs` for recursive mode** — strip directory components from
+  output paths.
+- [ ] **Notification hooks** — `--on-error` with richer environment variables
+  (exit code, error type, checksum).
+- [ ] **Retry with backoff visualization** — show retry attempts in progress
+  bar (attempt 2/3, waiting 4s...).
+- [ ] **Idempotent `--resume`** — auto-detect whether resume is possible
+  without manual flag.
+- [ ] **`--show-metadata`** — display `.goget.meta` contents for debugging.
+
+### 🔵 Lower Priority
+
+- [ ] **URL glob expansion (`--glob`)** — expand `[1-3]` and `{a,b}` patterns
+  in URLs.
+
+---
+
+## v1.3.0 — Security hardening
+
+TLS, authentication, and verification improvements.
+
+### 🔴 Supreme Priority
+
+- [ ] **PGP replacement** — `golang.org/x/crypto/openpgp` is frozen (archived
+  by Go team). Evaluate: inline a minimal subset, or wait for a community
+  `golang.org/x/` successor.
+
+### 🟡 High Priority
+
+- [ ] **`--pinned-cert` for all TLS protocols** — extend certificate pinning
+  to FTPS, SFTP, Gemini, and WebDAVS (currently HTTP/HTTPS only).
+- [ ] **SOCKS5 proxy for non-HTTP protocols** — FTP and SFTP through SOCKS5
+  proxy (currently HTTP/HTTPS only).
+- [ ] **DNS-over-HTTPS (DoH)** — optional DoH resolver. Must use only
+  `golang.org/x/` libraries.
+
+### 🟢 Medium Priority
+
+- [ ] **Structured error output with `--json`** — machine-readable error
+  objects with type, message, URL, and hint fields.
+
+---
+
+## v2.0.0 — New protocols
+
+Cloud providers, additional network protocols, and long-lived services.
+
+### 🔵 Lower Priority
+
+- [ ] **S3** — generic S3-compatible protocol with signature v4 auth
+  (covers AWS S3, MinIO, and any S3-compatible storage).
+- [ ] **Huawei OBS** — Huawei Cloud Object Storage.
+- [ ] **Alibaba OSS** — Alibaba Cloud Object Storage Service.
+- [ ] **Tencent COS** — Tencent Cloud Object Storage.
+- [ ] **Rsync** — incremental file transfer. Written from scratch
+  (rolling checksum, delta transfer, rsync wire protocol).
+- [ ] **TFTP** — trivial file transfer, useful for embedded devices and PXE.
+- [ ] **NFS** — NFS v3/v4 client. Written from scratch
+  (ONC RPC/XDR, portmapper, mount protocol, NFS protocol).
+- [ ] **goget-as-a-service** — daemon mode with REST API for remote download
+  management.
+
+---
+
+## v2.1.0 — Extensibility & UI
+
+### 🔵 Lower Priority
+
+- [ ] **Plugin system** — protocol handlers as external plugins.
+- [ ] **TUI (Terminal User Interface)** — interactive download manager with
+  live queue, progress, and log views. Keyboard navigation, color-coded
+  status, inline log viewer.
+- [ ] **Web GUI** — browser-based download manager (Vue 3 frontend, REST API,
+  live progress via SSE).
+- [ ] **Download speed test / benchmark mode** — measure maximum throughput
+  against a target server without saving the file.
+
+---
+
+## 💡 Backlog (unscheduled)
+
+Items that need more design, depend on external factors, or are
+lower priority than the versioned roadmap.
+
+### Performance
+
+- [ ] **Memory-mapped I/O for large files** — `mmap` for checksum
+  verification of multi-gigabyte downloads.
+- [ ] **Zero-copy for `file://` and `data:` protocols** — `sendfile(2)` /
+  `splice(2)` where available on Linux.
+- [ ] **HTTP/3 (QUIC)** — when Go stdlib gains QUIC support.
+
+### Advanced UX
+
+- [ ] **Scheduling — pause/resume queued downloads** — ability to pause the
+  queue, reorder items, and resume later.
+
+### Code Quality & Testing
+
+- [ ] **Integration test matrix** — CI against real FTP, SFTP, and WebDAV
+  servers.
+- [ ] **Coverage threshold increase** — raise from 40% to 60%.
+- [ ] **Benchmark suite** — `go test -bench` in CI.
+- [ ] **Fuzzing corpus in CI** — store and replay past crashes automatically.
+- [ ] **`internal/log` migration** — migrate remaining ~140 `fmt.Fprintf` and
+  `cli.Print*` callsites to `log.Infof`.
+- [ ] **`unsafe` audit** — verify no `unsafe` usage outside build-tag-gated
+  platform-specific code.
+
+### Project Infrastructure
+
+- [ ] **RPM spec file** — Fedora / RHEL / openEuler packaging.
+- [ ] **FreeBSD port** — official FreeBSD ports tree entry.
+- [ ] **systemd unit** — for daemon / scheduled mode.
+
+### Documentation
+
+- [ ] **Man page packaging** — already generated by `--generate-man-page`,
+  needs installation via packaging.
+- [ ] **Architecture decision records** — document why specific choices were
+  made (no short flags, IPv6-first, long flags only, no global state).
+- [ ] **Protocol-specific guides** — one doc per protocol with examples.
+- [ ] **Troubleshooting guide** — common errors and solutions.