Downloaders / Sync
Metalink support
Curl has it. Wget supports it.
Aria2
All-new 1.16 series debuted - aria2 aria2 - CLI Metalink/BitTorrent Client tatsuhiro-t/aria2 · GitHub
aria2c –split=5 –max-connection-per-server=5 http://www.savioko.com/100M.iso
A program to control aria2c xml-rpc in terminal lainme/aria2cmd · GitHub
GUI (The best ones)
YAAW by binux TheBear | Free Security & Utilities software downloads at SourceForge.net
Others
RSync & ZSync
zsync cm3/azsync/recursive-zsync at master · outofjungle/cm3 · GitHub cm3/azsync at master · outofjungle/cm3 · GitHub zsync: client-side rsync over HTTP - azure sky
ZSync block size
Default is 2K (2048) or 4K (4096), depending on file size. The maximum value allowed was 128KB (both in rsync and zsync).
I will use 65536 (64K). This is not a must, but it’s logical to be most efficient.
RSync Optimized Packers
GZip
Option –rsyncable in gzip.
When using with Tar, you can do even this:
GZIP="–rsyncable" tar -czf something.tgz /something
Parallel GZip = Pigz
It also supports –rsyncable parameter.
Parallel gzip Pigz – parallel gzip OMG | Kitchen Soap
Usage
Compress:
pigz --best --rsyncable --recursive --keep --blocksize 64
Decompress:
pigz --decompress testing.gz
I will use 64K as block size. Pigz minimal block size is 32K.
BZIP2 & Parallel BZIP2
By architecture (working in blocks), both of them are good for rsyncing.
Block size: bzip2 compresses large files in blocks. The flags -1 through -9 specify the block size to be 100,000 bytes through 900,000 bytes (the default) respectively.
Others
Anything in parallel Parallel xargs, but executes in parallel xjobs utility
Courgette (Google Chrome on Windows uses it): Courgette algorithm is a differential compression algorithm designed for incremental update of Google Chrome. Unlike other diff algorithms such as xdelta, zdelta and bsdiffwhich handle general binary files, the Courgette algorithm was designed to difference executable files supported by x8086 architecture. It achieves much smaller diff size.
Postoji i LFTP, koji ne zaostaje mnogo za rclone
LFTP supports a wide range of protocols, including plain vanilla FTP, along with FTPS, SFTP, HTTP, HTTPS, HFTP, FXP (see the “FXP” box), Fish, and BitTorrent.