petrbalvin/goget
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

docs: sync documentation with v1.1.0 protocol and CLI changes

Browse Source
This commit is contained in:
petrbalvin
2026-06-30 20:43:00 +02:00
parent 185e6ec20a
commit 21ac1f7487
8 changed files with 409 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+38 -40
View File
@@ -6,39 +6,40 @@
```mermaid
graph TD
User[User / Terminal]:::accent6
Script[Script / CI]:::accent6
CLI[cmd/goget/ - Main entry]:::accent0
API[pkg/api/ - Library interface]:::accent3
Core[internal/core/ - Types, Protocol Interface, Errors]:::accent1
Registry[internal/protocol/ - Protocol Registry]:::accent1
HTTP[internal/protocol/http/]:::accent1
FTP[internal/protocol/ftp/]:::accent1
SFTP[internal/protocol/sftp/]:::accent1
FILE[internal/protocol/file/]:::accent1
DataURL[internal/protocol/dataurl/]:::accent1
Gopher[internal/protocol/gopher/]:::accent1
Gemini[internal/protocol/gemini/]:::accent1
WebDAV[internal/protocol/webdav/]:::accent1
Transport[internal/transport/ - Dialer, Proxy, TLS, Rate Limit]:::accent7
Config[internal/config/]:::accent7
Output[internal/output/ - Atomic write, Resume, WARC]:::accent2
Auth[internal/auth/ - Basic, Digest, OAuth, netrc]:::accent7
Crypto[internal/crypto/ - Checksum, PGP, Certs]:::accent7
Cookie[internal/cookie/ - Netscape jar]:::accent7
HSTS[internal/hsts/ - RFC 6797 cache]:::accent7
Archive[internal/archive/ - Extraction]:::accent7
Recursive[internal/recursive/ - Crawler, Parser, Filter]:::accent2
Mirror[internal/mirror/]:::accent2
Metalink[internal/metalink/]:::accent2
Queue[internal/queue/]:::accent2
Compression[internal/compression/ - gzip, bzip2, zlib, flate, lzw]:::accent7
Log[internal/log/ - Structured logging]:::accent7
WARC[internal/warc/ - Archiving]:::accent7
Register[internal/protocol/register/ - Single registration]:::accent7
Format[internal/format/ - Bytes, Speed formatting]:::accent7
LinkRewrite[internal/linkrewrite/ - HTML link conversion]:::accent7
CLIUtil[internal/cli/ - Help, Progress bar, Colors]:::accent7
User[User / Terminal]
Script[Script / CI]
CLI[cmd/goget/ - Main entry]
API[pkg/api/ - Library interface]
Core[internal/core/ - Types, Protocol Interface, Errors]
Registry[internal/protocol/ - Protocol Registry]
HTTP[internal/protocol/http/]
FTP[internal/protocol/ftp/]
SFTP[internal/protocol/sftp/]
FILE[internal/protocol/file/]
DataURL[internal/protocol/dataurl/]
Gopher[internal/protocol/gopher/]
Gemini[internal/protocol/gemini/]
WebDAV[internal/protocol/webdav/]
Transport[internal/transport/ - Dialer, Proxy, TLS, Rate Limit]
Config[internal/config/]
Output[internal/output/ - Atomic write, Resume, WARC]
Auth[internal/auth/ - Basic, Digest, OAuth, netrc]
Crypto[internal/crypto/ - Checksum, PGP, Certs]
Cookie[internal/cookie/ - Netscape jar]
HSTS[internal/hsts/ - RFC 6797 cache]
Archive[internal/archive/ - Extraction]
Recursive[internal/recursive/ - Crawler, Parser, Filter]
Mirror[internal/mirror/]
Metalink[internal/metalink/]
Queue[internal/queue/]
Compression[internal/compression/ - gzip, bzip2, zlib, flate, lzw]
Log[internal/log/ - Structured logging]
WARC[internal/warc/ - Archiving]
Register[internal/protocol/register/ - Single registration]
Format[internal/format/ - Bytes, Speed formatting]
LinkRewrite[internal/linkrewrite/ - HTML link conversion]
CLIUtil[internal/cli/ - Help, Progress bar, Colors]
Pool[internal/pool/ - Unified worker pool]
User --> CLI
Script --> API
@@ -65,7 +66,10 @@ graph TD
HTTP --> HSTS
HTTP --> Output
FTP --> Transport
FTP --> Pool
SFTP --> Transport
SFTP --> Pool
WebDAV --> Pool
end
subgraph "Post-Download"
@@ -88,13 +92,6 @@ graph TD
Transport --> Auth
Transport --> Config
classDef accent0 fill:#3B82F6,stroke:#2563EB,color:#fff
classDef accent1 fill:#22C55E,stroke:#16A34A,color:#fff
classDef accent2 fill:#F59E0B,stroke:#D97706,color:#000
classDef accent3 fill:#A855F7,stroke:#7C3AED,color:#fff
classDef accent6 fill:#6366F1,stroke:#4F46E5,color:#fff
classDef accent7 fill:#64748B,stroke:#475569,color:#fff
```
## Layers
@@ -168,6 +165,7 @@ Protocols register themselves by scheme in a global `Registry`. Scheme normaliza
| `log` | Structured logger with text and JSON output modes, level filtering, ANSI colors |
| `warc` | WARC (Web ARChive) format writing for legal and archival compliance |
| `cli` | ANSI color output, progress bar, help text, completion |
| `pool` | Protocol-agnostic recursive worker pool — bounded goroutine execution with recursive task submission. Replaces per-protocol semaphore+WaitGroup code in FTP, SFTP, and WebDAV recursive-parallel downloads |
## Download Flow