How to use download compression in wget

Download compression in wget reduces bandwidth consumption and accelerates transfers for text-heavy responses such as JSON APIs, log exports, and configuration snapshots over slow or metered links. Smaller on-the-wire payloads keep automation-friendly downloads efficient while still producing complete files on disk.

HTTP content encoding relies on the Accept-Encoding header to announce supported formats like gzip or deflate. When compression is enabled, wget negotiates an encoding with the server, transparently decompresses the response stream as it arrives, and writes the restored data to the output file so the saved content matches the original resource format.

Not every origin, proxy, or CDN supports the same encodings, and some legacy stacks handle less common schemes poorly or disable compression for large objects. Compression trades CPU cycles for reduced bandwidth, which benefits high-latency or constrained links more than fast local networks. Modern builds of wget typically provide the --compression option and related headers, but confirming support and testing a sample transfer prevents surprises in scripts and cron jobs.

1. Open a terminal on a system where wget is available.

   $ whoami
   user

2. Display the installed wget version to confirm support for the --compression option.

   $ wget --version | head -n 3
   GNU Wget 1.21.4 built on linux-gnu.
   
   -cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls 

   Releases from GNU Wget 1.20 and newer typically include automatic HTTP content-encoding negotiation.

3. Enable automatic compression negotiation during a download to a JSON file.

   $ cd ~/ && wget --compression=auto https://www.example.com/api/data -O data.json
   --2026-01-10 04:09:12--  https://www.example.com/api/data
   Resolving www.example.com (www.example.com)... 203.0.113.50
   Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 75 [application/json]
   Saving to: 'data.json'
   
        0K                                                       100% 3.87M=0s
   
   2026-01-10 04:09:12 (3.87 MB/s) - 'data.json' saved [65]

   The --compression=auto setting adds an appropriate Accept-Encoding header and writes decompressed content to data.json.

4. Inspect the HTTP response headers to verify that the server used compression on the wire.

   $ cd ~/ && wget --compression=auto --server-response --spider https://www.example.com/api/data 2>&1 | grep -i content-encoding
     Content-Encoding: gzip

   A Content-Encoding value such as gzip or deflate indicates that compressed transfer was negotiated successfully.

5. Override the encoding list with a custom Accept-Encoding header when a service requires specific values.

   $ cd ~/ && wget --compression=auto --header="Accept-Encoding: gzip, deflate" https://www.example.com/api/data -O data.json
   --2026-01-10 04:11:42--  https://www.example.com/api/data
   Resolving www.example.com (www.example.com)... 203.0.113.50
   Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 75 [application/json]
   Saving to: 'data.json'
   
        0K                                                       100% 2.47M=0s
   
   2026-01-10 04:11:42 (2.47 MB/s) - 'data.json' saved [65]

   Using unsupported or incorrect Accept-Encoding combinations can trigger HTTP errors, disable compression, or cause intermediaries such as proxies and CDNs to behave unpredictably.

6. Download a pre-compressed artifact such as a .json.gz export without altering the on-disk format.

   $ cd ~/ && wget https://www.example.com/api/data.json.gz -O data.json.gz
   --2026-01-10 04:11:47--  https://www.example.com/api/data.json.gz
   Resolving www.example.com (www.example.com)... 203.0.113.50
   Connecting to www.example.com (www.example.com)|203.0.113.50|:443... connected.
   HTTP request sent, awaiting response... 200 OK
   Length: 75 [application/gzip]
   Saving to: 'data.json.gz'
   
        0K                                                       100% 5.03M=0s
   
   2026-01-10 04:11:47 (5.03 MB/s) - 'data.json.gz' saved [75/75]

   Pre-compressed artifacts remain compressed on disk and can be processed later with tools such as gunzip or tar.

7. Confirm that the saved JSON file is stored uncompressed on disk while the transfer still used HTTP compression.

   $ cd ~/ && file data.json && wc -c data.json
   data.json: JSON text data
   65 data.json

   A plain-text classification such as JSON text data from file combined with the earlier Content-Encoding header and the byte count from wc verifies that decompression occurred locally while the network payload stayed compact.