How to use download compression in cURL

Download compression reduces HTTP response size by encoding data with algorithms such as gzip, deflate, brotli, or zstd before transmission. Smaller payloads improve transfer times and decrease bandwidth usage, which is especially useful for large JSON, HTML, and other text-heavy responses fetched over slow or congested links.

During an HTTP request, a client advertises supported encodings in the Accept-Encoding header, and the server responds with a Content-Encoding header describing the chosen algorithm. With option --compressed, cURL automatically sends a suitable Accept-Encoding header for common encodings and transparently decompresses the response body before displaying or saving it.

Compression effectiveness depends on server configuration, intermediary proxies, and the nature of transferred data. Some servers ignore encoding requests, some only offer a subset of algorithms, and media formats that are already compressed gain little benefit, so potential CPU overhead and troubleshooting of mismatched encodings should be considered when enabling download compression in performance-sensitive environments.

1. Open a terminal where cURL is available and confirm that content-encoding support is compiled in.

   $ curl --version
   curl 8.5.0 (aarch64-unknown-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
   Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.6
   Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
   Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
   ##### snipped #####

   Presence of zlib, brotli, or zstd in the feature list indicates built-in support for compressed HTTP responses.

2. Request a resource with automatic negotiation and decompression by using --compressed.

   $ curl --silent --compressed "https://www.example.com/data.json"
   {
     "status": "ok",
     "items": [
       {"id": 1, "name": "alpha"},
       {"id": 2, "name": "beta"}
     ]
   }

   Option --compressed adds an Accept-Encoding header for common algorithms and decodes the compressed response body before printing it.

3. Store the decompressed response directly to a file for later processing.

   $ curl --silent --compressed "https://www.example.com/data.json" --output data.json

   Saving decoded content allows downstream tools to read data.json without additional decompression steps.

4. Request explicitly compressed data and keep it encoded for external tools or pipelines.

   $ curl --silent --header "Accept-Encoding: gzip" "https://www.example.com/data.json" --output data.json.gz
   $ gunzip data.json.gz
   $ head -n 3 data.json
   {
     "status": "ok",
     "items": [
   }

   Explicit Accept-Encoding values retain the raw gzip stream so that utilities such as gunzip or zstd can handle decompression within a larger processing pipeline.

5. Inspect verbose headers to confirm that compression is negotiated and applied.

   $ curl --compressed --silent -v -o /dev/null "https://www.example.com/data.json"
   ##### snipped #####
   > Accept-Encoding: deflate, gzip, br, zstd
   ##### snipped #####
   < content-encoding: gzip
   ##### snipped #####

   Success indicators include an Accept-Encoding request header, a Content-Encoding response header such as gzip, and reduced transfer size for text-based responses.

6. Avoid enabling download compression for formats that are already compressed.

   $ curl --silent --compressed "https://www.example.com/archive.zip" --output archive.zip

   Applying HTTP compression to .zip, .jpg, or MP4 files increases CPU usage on client and server without meaningful size reduction and can slightly worsen transfer latency.