feat: initial goget release — modern IPv6-first download utility
This commit is contained in:
@@ -0,0 +1,258 @@
|
||||
# Security
|
||||
|
||||
goget implements multiple security features to protect both the client and server during file transfers. This document covers TLS configuration, certificate handling, checksum verification, PGP support, HSTS, and SSRF protection.
|
||||
|
||||
## TLS / HTTPS
|
||||
|
||||
### Default Security
|
||||
|
||||
- **TLS 1.2 minimum** — TLS 1.0 and 1.1 are disabled
|
||||
- **Certificate verification** — Enabled by default against system CA bundle
|
||||
- **HTTP/2 support** — Automatic ALPN negotiation
|
||||
|
||||
### Certificate Pinning
|
||||
|
||||
Pin a specific certificate by its SHA-256 hash:
|
||||
|
||||
```bash
|
||||
goget --pinned-cert "a1b2c3d4e5f6..." --url https://example.com
|
||||
```
|
||||
|
||||
If the server certificate's SHA-256 doesn't match, the connection is aborted.
|
||||
|
||||
### Client Certificates (mTLS)
|
||||
|
||||
Authenticate the client to the server with mutual TLS:
|
||||
|
||||
```bash
|
||||
goget --cert client.pem --key client.key --url https://mtls.example.com
|
||||
```
|
||||
|
||||
### Custom CA Bundle
|
||||
|
||||
```bash
|
||||
goget --cacert custom-ca.pem --url https://internal.example.com
|
||||
```
|
||||
|
||||
### Insecure Mode
|
||||
|
||||
Disable all TLS verification (⚠ **for testing only**):
|
||||
|
||||
```bash
|
||||
goget --insecure --url https://self-signed.local
|
||||
```
|
||||
|
||||
### SSLKEYLOGFILE
|
||||
|
||||
Log TLS session keys for Wireshark debugging:
|
||||
|
||||
```bash
|
||||
goget --ssl-key-log tls.keys --url https://example.com
|
||||
# Then: Wireshark → Preferences → TLS → (Pre-)Master-Secret log filename → tls.keys
|
||||
```
|
||||
|
||||
## HSTS (HTTP Strict Transport Security)
|
||||
|
||||
Per RFC 6797, goget maintains an HSTS cache that automatically upgrades HTTP connections to HTTPS for known hosts.
|
||||
|
||||
### How It Works
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client as goget
|
||||
participant Server as Server
|
||||
|
||||
Client->>Server: GET http://example.com
|
||||
Server-->>Client: 200 OK + Strict-Transport-Security: max-age=31536000
|
||||
Client->>Client: Store: example.com → 365 days
|
||||
Client->>Server: GET http://example.com/page (later)
|
||||
Client->>Client: HSTS cache hit → upgrade to HTTPS
|
||||
Client->>Server: GET https://example.com/page
|
||||
```
|
||||
|
||||
### Cache Location
|
||||
|
||||
```
|
||||
~/.config/goget/hsts
|
||||
```
|
||||
|
||||
Can be overridden with `GOGET_HSTS` environment variable or `--hsts-file`.
|
||||
|
||||
### Cache Format
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"host": "example.com",
|
||||
"include_subdomains": true,
|
||||
"expires": "2027-06-01T00:00:00Z"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Expired entries are automatically pruned on load.
|
||||
|
||||
## Checksum Verification
|
||||
|
||||
Verify file integrity after download:
|
||||
|
||||
### Direct Hash
|
||||
|
||||
```bash
|
||||
goget --checksum "a1b2c3d4e5f6..." --checksum-algo sha256 --url https://example.com/file.zip
|
||||
```
|
||||
|
||||
Supported algorithms:
|
||||
|
||||
| Algorithm | Flag Value | Hash Length |
|
||||
|---|---|---|
|
||||
| SHA-256 | `sha256` | 64 hex chars |
|
||||
| SHA-512 | `sha512` | 128 hex chars |
|
||||
| SHA3-256 | `sha3-256` | 64 hex chars |
|
||||
| SHA3-512 | `sha3-512` | 128 hex chars |
|
||||
| BLAKE2b | `blake2b` | 128 hex chars |
|
||||
| MD5 | `md5` | 32 hex chars |
|
||||
|
||||
### Checksum File
|
||||
|
||||
Supports standard `SHA256SUMS` format:
|
||||
|
||||
```bash
|
||||
# SHA256SUMS file:
|
||||
# a1b2c3d4... file.zip
|
||||
# e5f6a7b8... *file.tar.gz
|
||||
|
||||
goget --checksum-file SHA256SUMS --url https://example.com/file.zip
|
||||
```
|
||||
|
||||
Both plain (`file.zip`) and binary-mode (`*file.tar.gz`) formats are supported.
|
||||
|
||||
### Error Handling
|
||||
|
||||
On mismatch:
|
||||
```
|
||||
Checksum mismatch
|
||||
Expected: a1b2c3d4...
|
||||
Actual: e5f6a7b8...
|
||||
```
|
||||
|
||||
The error is a typed `core.GogetError` with `Type: "CHECKSUM"` and `Details["expected"]` / `Details["actual"]`.
|
||||
|
||||
## PGP / GPG
|
||||
|
||||
### Signature Verification
|
||||
|
||||
Verify PGP detached signatures:
|
||||
|
||||
```bash
|
||||
# With explicit signature file
|
||||
goget --pgp-verify --pgp-sig file.sig --pgp-key public.key --url https://example.com/file
|
||||
|
||||
# Auto-detect signature file (tries: file.asc, file.sig, file.gpg)
|
||||
goget --pgp-verify --pgp-key public.key --url https://example.com/file
|
||||
```
|
||||
|
||||
### Decryption
|
||||
|
||||
Decrypt PGP-encrypted files:
|
||||
|
||||
```bash
|
||||
# Decrypt with key and passphrase
|
||||
goget --pgp-decrypt --pgp-key private.key --pgp-passphrase "secret" --url https://example.com/file.gpg
|
||||
|
||||
# Read passphrase from file (safer)
|
||||
goget --pgp-decrypt --pgp-key private.key --pgp-passphrase-file pass.txt --url https://example.com/file.gpg
|
||||
```
|
||||
|
||||
The decrypted file replaces the encrypted one:
|
||||
- `file.gpg` → `file`
|
||||
- `file.pgp` → `file`
|
||||
- `file.asc` → `file`
|
||||
- Other → `filename.decrypted`
|
||||
|
||||
> **Note:** PGP functionality uses `golang.org/x/crypto/openpgp`, which is frozen. A replacement will be provided in a future release.
|
||||
|
||||
## SSRF Protection
|
||||
|
||||
By default, goget blocks connections to private and internal IP ranges:
|
||||
|
||||
| Range | CIDR | Description |
|
||||
|---|---|---|
|
||||
| `10.0.0.0/8` | Private | RFC 1918 |
|
||||
| `172.16.0.0/12` | Private | RFC 1918 |
|
||||
| `127.0.0.0/8` | Loopback | Internal |
|
||||
| `169.254.0.0/16` | Link-local | Auto-configuration |
|
||||
| `::1/128` | IPv6 loopback | Internal |
|
||||
|
||||
### IPv4-Mapped IPv6 Normalization
|
||||
|
||||
DNS-controlled IPv4-mapped IPv6 addresses (e.g. `::ffff:10.0.0.1`) are now normalized to plain IPv4 before the private-IP check. This prevents an attacker from reaching internal hosts through DNS responses that embed IPv4 addresses in IPv6 form. Covered by a 17-case unit test in the transport layer.
|
||||
|
||||
Override with `--allow-private`:
|
||||
|
||||
```bash
|
||||
goget --allow-private --url https://internal.example.com
|
||||
```
|
||||
|
||||
## DNS Security
|
||||
|
||||
### Custom Resolvers
|
||||
|
||||
Bypass system DNS for specific queries:
|
||||
|
||||
```bash
|
||||
goget --dns-servers 1.1.1.1,8.8.8.8 --url https://example.com
|
||||
```
|
||||
|
||||
### IPv6-First
|
||||
|
||||
The transport dialer prefers IPv6 and only falls back to IPv4 when no IPv6 address is available. This reduces NAT-related attack surface:
|
||||
|
||||
```bash
|
||||
# IPv6 only (abort if no IPv6)
|
||||
goget --no-ipv4 --url https://example.com
|
||||
```
|
||||
|
||||
## Credential Safety
|
||||
|
||||
### Safe Error Logging
|
||||
|
||||
When a URL contains user credentials (`https://user:pass@example.com`), `core.SafeURL()` strips them from error messages and logs:
|
||||
|
||||
```go
|
||||
// Original: https://admin:secret@api.example.com/data
|
||||
// Logged: https://api.example.com/data
|
||||
```
|
||||
|
||||
### OAuth Error Sanitization
|
||||
|
||||
Non-2xx OAuth responses are sanitized to extract only the structured `error` / `error_description` fields (RFC 6749 §5.2). Providers that echo `client_secret` or other secrets in their error response bodies can no longer leak them into goget logs or error displays.
|
||||
|
||||
### Config Sanitization
|
||||
|
||||
`config.Save()` strips sensitive fields before writing to disk:
|
||||
- `auth.password`
|
||||
- `auth.oauth.client_secret`
|
||||
- `auth.oauth.access_token`
|
||||
- `auth.oauth.refresh_token`
|
||||
|
||||
### File Permission Hardening
|
||||
|
||||
Configuration files (`config.toml`), resume metadata sidecars (`.goget.meta`), and atomic temp files are written with mode `0600` (owner-only), so OAuth tokens, source URLs, and downloaded payloads are never world-readable on multi-user systems.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `GOGET_CONFIG` — alternative config file path
|
||||
- `GOGET_PASSWORD` — password override (no file storage)
|
||||
- `GOGET_HSTS` — alternative HSTS cache path
|
||||
- `NO_COLOR` — disable color output
|
||||
|
||||
## Timeouts & Limits
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---|---|---|
|
||||
| Connection timeout | 30s (auto) | Smart calculation based on file size |
|
||||
| Max total time | — (`--max-time`) | Abort if exceeded |
|
||||
| Max file size | — (`--max-filesize`) | Abort if server response exceeds limit |
|
||||
| Max retries | 3 | Exponential backoff with 30s cap |
|
||||
| Retry delay | 1s initial, 2× multiplier | Configurable via `--retry-delay` |
|
||||
Reference in New Issue
Block a user