goget/docs/authentication.md

Authentication

goget supports multiple authentication mechanisms for HTTP, FTP, and SFTP connections. Credentials are resolved in priority order, with .netrc auto-reading as a fallback.

Priority Order

1. CLI flags (--username, --password, --password-file)
2. OAuth 2.0 tokens (if configured via CLI or config)
3. .netrc auto-reading (~/.netrc)
4. Anonymous / no auth

HTTP Basic Authentication (RFC 7617)

The simplest mechanism. Credentials are Base64-encoded in the Authorization header.

goget --username admin --password secret --url https://api.example.com/data

Authorization: Basic YWRtaW46c2VjcmV0

Security Notes

• Basic auth sends credentials in plain text (Base64 is not encryption)
• Always use HTTPS with Basic auth
• Prefer --password-file for scripting to avoid credentials in shell history:

goget --username admin --password-file ./secret.txt --url https://api.example.com

HTTP Digest Authentication (RFC 7616)

Digest auth never sends the password in plain text. Instead, it uses a challenge-response mechanism with MD5 or SHA-256 hashing:

sequenceDiagram
    participant Client
    participant Server

    Client->>Server: GET /protected
    Server-->>Client: 401 + WWW-Authenticate: Digest nonce="abc", realm="app"
    Client->>Server: GET /protected + Authorization: Digest response="hash"
    Server-->>Client: 200 OK

Usage

goget --username admin --password secret --auth-type digest --url https://api.example.com
goget --username admin --password secret --auth-type auto --url https://api.example.com

With --auth-type auto, goget first tries to make the request. If the server returns 401 with a WWW-Authenticate: Digest header, it automatically switches to Digest auth.

Implementation

The internal/auth package implements Digest auth per RFC 7616:

• HA1 = MD5(username:realm:password)
• HA2 = MD5(method:digestURI)
• response = MD5(HA1:nonce:nc:cnonce:qop:HA2)

The cnonce (client nonce) is generated using crypto/rand for each request.

OAuth 2.0

goget supports four OAuth 2.0 grant types:

Client Credentials Grant

For server-to-server communication without user interaction:

goget \
  --oauth-client-id "my-client" \
  --oauth-client-secret "my-secret" \
  --oauth-token-url "https://auth.example.com/token" \
  --oauth-grant-type client_credentials \
  --oauth-scopes "read,write" \
  --url https://api.example.com/data

Authorization Code Grant

For web applications with user interaction:

# Step 1: Generate authorization URL
goget \
  --oauth-client-id "my-client" \
  --oauth-auth-url "https://auth.example.com/authorize" \
  --oauth-redirect-uri "http://localhost:8080/callback" \
  --url https://api.example.com/data

# Step 2: Exchange code for token (after user authorizes)
goget \
  --oauth-client-id "my-client" \
  --oauth-client-secret "my-secret" \
  --oauth-token-url "https://auth.example.com/token" \
  --oauth-grant-type authorization_code \
  --url https://api.example.com/data

Refresh Token Grant

For maintaining long-lived access:

goget \
  --oauth-client-id "my-client" \
  --oauth-client-secret "my-secret" \
  --oauth-token-url "https://auth.example.com/token" \
  --oauth-refresh-token "ref_xxx" \
  --oauth-grant-type refresh_token \
  --url https://api.example.com/data

Token Management

client := auth.NewOAuthClient(cfg)

// Check if token is valid (not expired)
if !client.IsTokenValid() {
    // Refresh automatically
    client.RefreshToken(ctx)
}

// Apply token to HTTP request
client.ApplyToken(httpReq)
// → Authorization: Bearer eyJhbG...

The token expiry is tracked with a 30-second grace period.

Security

• Token URL enforced to HTTPS — plain HTTP token endpoints are rejected unless they're localhost or 127.0.0.1
• Tokens never logged — The config Save() method strips access_token, refresh_token, and client_secret before writing to disk

.netrc Auto-Reading

When no explicit credentials are provided via CLI flags or OAuth config, goget automatically checks ~/.netrc:

machine api.example.com
    login admin
    password secret

machine default
    login guest
    password guest123

Usage

# No --username or --password needed if ~/.netrc has a matching entry
goget --url https://api.example.com/data

Resolution Order

1. Exact hostname match — api.example.com matches machine api.example.com
2. Default entry — machine default used as fallback
3. No match — proceeds without credentials

Skip .netrc

Use --no-private to disable .netrc and auth file reading:

goget --no-private --url https://api.example.com

Cookies

goget supports Netscape-format cookie jars:

CLI Cookies

goget --cookie "session=abc123" --cookie "theme=dark" --url https://example.com

Cookie Jar

Persist cookies across sessions:

# Save cookies to file
goget --cookie-jar cookies.txt --url https://example.com

# Load cookies from file
goget --cookie-jar cookies.txt --url https://example.com/protected

The cookie jar uses the standard Netscape format:

SFTP Authentication

SFTP connections authenticate via SSH:

SSH Key Authentication

goget auto-detects keys from standard locations:

• ~/.ssh/id_rsa
• ~/.ssh/id_ecdsa
• ~/.ssh/id_ed25519

goget --url sftp://user@host:/path/to/file.zip

Password Authentication

goget --url sftp://user@host:/path/to/file.zip --password "secret"

Known Hosts

Host key verification uses ~/.ssh/known_hosts by default:

# Custom known_hosts file
goget --url sftp://user@host:/path/to/file.zip --known-hosts ~/.ssh/known_hosts

# Skip verification (first connection)
goget --url sftp://user@host:/path/to/file.zip --ssh-insecure

FTP Authentication

FTP connections default to anonymous login. For authenticated FTP:

# Username/password
goget --url ftp://ftp.example.com/file.zip --username user --password pass

# Anonymous (default)
goget --url ftp://anonymous@ftp.example.com/file.zip

Access Tokens in URLs

While https://user:pass@example.com URLs are supported, prefer --username and --password CLI flags. When a URL contains credentials, they're automatically stripped from error messages and logs via core.SafeURL().