Goget — Modern IPv6-First Command-Line Download Utility

goget is a modern command-line download utility built on the Go standard library. It provides IPv6-first connections, parallel chunked downloads, resume support, checksum verification, recursive website mirroring, and support for 9 network protocols — all in a single static binary with zero runtime dependencies.

It is designed to be:

• IPv6-first with automatic IPv4 fallback — always connects via the fastest available path.
• Safe by default — TLS 1.2 minimum, certificate verification, SSRF protection, and HSTS cache (RFC 6797).
• Self-contained — one static binary, zero runtime dependencies beyond the Go standard library and golang.org/x/ packages.
• Scriptable — JSON progress output, --write-out metadata, post-download hooks, and a public Go library API.

---

Stack

Concern	Choice
Language	Go (≥ 1.26)
Protocols	HTTP/1.1, HTTP/2, FTP, SFTP, WebDAV, Gemini, Gopher
HTTP transport	Custom dialer with IPv6-first resolution
TLS	Go crypto/tls (1.2 minimum)
Markup	golang.org/x/net/html for recursive downloads
Crypto	stdlib + golang.org/x/crypto for PGP

---

Features

• 9 protocols — HTTP/1.1, HTTP/2, FTP (active/passive, explicit TLS), SFTP with known_hosts verification, WebDAV (webdav://, webdavs://) with recursive directory download and upload, Gemini, Gopher, data:, file://.
• Parallel chunked downloads — automatically splits files over 100 MB into concurrent chunks using byte-range requests.
• Resume interrupted transfers — metadata-driven resume with .goget.meta tracking across HTTP, FTP, SFTP, WebDAV, and Gemini.
• Recursive downloading & mirroring — follow links, filter by pattern or MIME type, limit depth, stay within domain. Full site mirrors with link conversion for offline viewing and robots.txt compliance.
• Checksum verification — SHA-256, SHA-512, SHA3-256, SHA3-512, BLAKE2b, MD5; auto-parse SHA256SUMS files.
• PGP support — detached signature verification and OpenPGP decryption.
• Metalink — multi-source downloads with automatic failover (.meta4, .metalink).
• Download queue — persistent queue with priority (1–10).
• Scheduling — absolute datetime or cron expressions.
• OAuth 2.0 — client credentials, authorization code, and refresh token flows.
• Proxy support — HTTP CONNECT, SOCKS5 (socks5://, socks5h://).
• HSTS cache — RFC 6797 compliance, automatic HTTP→HTTPS upgrade.
• WARC output — Web ARChive format for legal and archival compliance.
• Library API — pkg/api/ package for programmatic downloads, uploads, and queue management.
• Archive extraction — TAR, TAR.GZ, TAR.BZ2, ZIP with auto-detection.
• Shell completion — bash, zsh, and fish.
• IDN support — international domain names with automatic Punycode conversion.

---

Quick Start

git clone https://codeberg.org/petrbalvin/goget && cd goget && just install && just run --version

just install downloads Go module dependencies. just build builds a static binary to bin/goget.

# Download a file
goget --url https://example.com/file.zip

# Parallel chunked download (auto-enabled for files >100 MB)
goget --url https://example.com/large.iso --parallel 4

# Resume interrupted download
goget --url https://example.com/file.zip --resume

# Multiple URLs
goget --url https://a.com/1.zip --url https://b.com/2.zip

# Batch download from file
goget --batch-file urls.txt

# Custom output directory
goget --url https://example.com/file.zip --output ./downloads/file.zip

---

Usage

Common Flags

# Custom HTTP headers
goget --header "Authorization: Bearer token" --url https://api.example.com

# Cookies
goget --cookie "session=abc123" --url https://example.com

# POST data
goget --data "key=value" --url https://httpbin.org/post

# Skip TLS verification (⚠ testing only)
goget --insecure --url https://self-signed.example.com

# Client certificate (mTLS)
goget --cert client.pem --key client.key --url https://mtls.example.com

# HTTP trace timing
goget --trace-time --url https://example.com
# → DNS: 12ms | TCP: 34ms | TLS: 56ms | TTFB: 78ms | Total: 180ms

# Timeouts
goget --max-time 30s --url https://slow.example.com
goget --connect-timeout 5s --url https://example.com

# Rate limiting
goget --rate-limit 1MB/s --url https://example.com/file.zip

# Proxy
goget --proxy http://proxy.example.com:8080 --url https://example.com/file.zip
goget --proxy socks5h://127.0.0.1:1080 --url https://example.com/file.zip

# Pin server certificate (applies to HTTPS, FTPS, Gemini, WebDAVS)
goget --pinned-cert a1b2c3d4e5f6... --url https://example.com/file
goget --pinned-cert a1b2c3d4e5f6... --url gemini://gemini.example.com/

# Pin SSH host key for SFTP (OpenSSH fingerprint from `ssh-keygen -lf`)
goget --pinned-host-key "SHA256:AAAAC3NzaC1lZDI1NTE5AAAAIE..." \
      --url sftp://example.com/file

Recursive & Mirroring

# Recursive download with depth limit
goget --url https://example.com/docs/ --recursive --max-depth 3

# Pattern filtering
goget --recursive --accept "*.pdf,*.html" --url https://example.com

# Dry-run — list without downloading
goget --url https://example.com/docs/ --recursive --dry-run

# Mirror entire website with link conversion
goget --url https://example.com --mirror --convert-links --output ./mirror

# WARC archiving
goget --warc-file archive.warc --url https://example.com/page.html

Verification

# Checksum
goget --checksum abc123... --url https://example.com/file.zip
goget --checksum-file SHA256SUMS --url https://example.com/file.zip

# PGP
goget --pgp-verify --pgp-sig file.sig --url https://example.com/file

# Metalink multi-source
goget --metalink --url https://example.com/release.meta4

Advanced

# Schedule a download
goget --schedule "2026-06-01 02:00" --url https://example.com/daily.zip
goget --schedule "0 2 * * *" --url https://example.com/daily.zip

# Post-download hook (env: GOGET_OUTPUT, GOGET_SIZE, GOGET_URL)
goget --on-complete "process.sh" --url https://example.com/data.csv

# JSON progress for scripting
goget --json --no-progress --url https://example.com/file.zip

# Inspect a partially-downloaded transfer's resume metadata
goget --show-metadata ./file.zip
goget --show-metadata ./file.zip.goget.meta --json

# Shell completion
goget --completion bash > /etc/bash_completion.d/goget
goget --completion zsh > /usr/share/zsh/site-functions/_goget
goget --completion fish > ~/.config/fish/completions/goget.fish

# Configuration management
goget --init-config
goget --config-get timeout
goget --config-set timeout 5m
goget --config-list

---

Configuration

Default location: ~/.config/goget/config.toml. See docs/configuration.md for the full schema.

timeout = "30m0s"
parallel = 0
user_agent = "Goget/1.0.0"
max_speed = 0
auto_resume = true
auto_decompress = true
insecure_skip_verify = false
checksum_algo = "sha256"
color = "auto"
progress_style = "unicode"
max_retries = 0
proxy = ""

[recursive]
enabled = false
max_depth = 3
follow_external = false
delay = "100ms"

---

Library Usage

package main

import (
    "fmt"
    "log"
    "time"

    "codeberg.org/petrbalvin/goget/pkg/api"
)

func main() {
    client := api.NewClient(
        api.WithTimeout(5 * time.Minute),
        api.WithUserAgent("MyApp/1.0"),
    )
    defer client.Close()

    result, err := client.Download(&api.DownloadRequest{
        URL:    "https://example.com/file.zip",
        Output: "output.zip",
        Resume: true,
    })
    if err != nil {
        log.Fatalf("download failed: %v", err)
    }
    fmt.Printf("Downloaded %d bytes in %v (%.1f MB/s, IPv%d)\n",
        result.BytesDownloaded, result.Duration,
        result.Speed/1024/1024, result.IPVersion)
}

Full API reference: docs/api.md.

---

Development

Requirements

• Go 1.26+
• Linux (amd64, arm64, riscv64, loong64) or FreeBSD (amd64, arm64, riscv64)

just              # list recipes
just install      # go mod download
just run          # run from source (development)
just build        # go vet + gofmt check + build → bin/goget
just test         # go test -race -count=1 ./...
just man          # validate man/goget.1 with groff
just install-man  # install manpage to ~/.local/share/man/man1/
just uninstall    # remove build artifacts
just uninstall-man# remove installed manpage

---

Documentation

• man/goget.1 — troff manual page (man goget after just install-man)
• docs/architecture.md — components and request flow
• docs/configuration.md — config schema (TOML)
• docs/api.md — public Go library API
• docs/cli-reference.md — full CLI flag reference
• docs/protocols.md — protocol handler details
• docs/recursive-mirror.md — recursive downloading and mirroring
• docs/download-pipeline.md — download flow
• docs/authentication.md — auth mechanisms
• docs/security.md — threat model and controls
• docs/migration.md — curl/wget → goget flag mapping
• docs/development.md — dev workflow, releases, CI

Contributing

See CONTRIBUTING.md for development setup, code style, commit conventions, the pull request flow, and how releases are cut.

License

MIT — see LICENSE.
Copyright © 2026 Petr Balvín