How to debug wget connections

Download failures are hard to fix when the only visible symptom is a generic timeout or HTTP error. A captured Wget debug trace exposes each stage of the connection path so DNS, TCP reachability, TLS, and application responses can be separated instead of guessed at.

The --debug option expands Wget logging with request and response details, TLS negotiation notes, and connection-state messages that do not appear in normal output. Pairing it with --output-file keeps the trace in a durable log file for repeated inspection, filtering, and comparison after the command exits.

Debug logs often contain full URLs, headers, redirects, and other context that should not be left in shared locations. Use a private directory, prefer short representative URLs, and switch to --spider when the goal is connection diagnosis rather than saving the payload itself.

1. Create a private directory for debug traces before capturing any connection data.

   $ mkdir -p "$HOME/wget-debug"
   $ chmod 700 "$HOME/wget-debug"

   Restricted permissions reduce the chance of leaking headers, URLs, or diagnostic details from the captured logs.

2. Capture one focused debug trace with --spider so the connection path can be inspected without saving a payload.

   $ wget --debug --output-file="$HOME/wget-debug/wget-403.log" --spider https://httpbin.org/status/403

   A short failing URL is easier to analyze than a long download that mixes connection noise with file-transfer progress.

3. Check the resolution and socket-establishment lines first to see whether the failure happens before HTTP response handling.

   $ rg -n 'Resolving|Connecting to|HTTP request sent|awaiting response' "$HOME/wget-debug/wget-403.log"
   8:Resolving httpbin.org (httpbin.org)... 44.221.213.41, 44.196.185.120, 52.71.170.232, ...
   10:Connecting to httpbin.org (httpbin.org)|44.221.213.41|:443... connected.
   29:HTTP request sent, awaiting response...

   Missing or broken Resolving lines point toward DNS problems, while failures after Connecting to usually belong to TLS or HTTP handling.

   Related: How to check DNS resolution in Linux

4. Check for proxy negotiation when the request is expected to go direct or when a proxy path is suspected.

   $ rg -ni '\bproxy\b|^CONNECT ' "$HOME/wget-debug/wget-403.log" || echo 'no proxy lines found'
   no proxy lines found

   Unexpected proxy or CONNECT lines usually trace back to environment variables such as http_proxy and https_proxy or to startup-file proxy directives.

   Related: How to use an HTTP proxy with wget

5. Inspect the TLS section before changing certificates, URLs, or retry settings.

   $ rg -n 'TLS|SSL|certificate|handshake' "$HOME/wget-debug/wget-403.log"
   13:Initiating SSL handshake.
   14:Handshake successful; connected socket 7 to SSL handle 0x00000009bd414000
   15:certificate:
   18:X509 certificate successfully verified and matches host httpbin.org

   Successful handshake and certificate lines prove that transport setup completed before the request reached the application layer.

   Use --no-check-certificate only as a temporary diagnostic on controlled systems because it disables server identity verification.

   Related: How to ignore SSL certificate errors in wget

6. Review the request block and the returned status code to decide whether the problem is transport or application policy.

   $ sed -n '/---request begin---/,/---request end---/p' "$HOME/wget-debug/wget-403.log"
   ---request begin---
   HEAD /status/403 HTTP/1.1
   Host: httpbin.org
   User-Agent: Wget/1.25.0
   Accept: */*
   Accept-Encoding: identity
   Connection: Keep-Alive
   ---request end---
   
   $ rg -n 'HTTP/|403|Location:' "$HOME/wget-debug/wget-403.log"
   21:HEAD /status/403 HTTP/1.1
   31:HTTP/1.1 403 FORBIDDEN
   41:403 FORBIDDEN

   A visible 403 or redirect chain usually points to URL, auth, or header policy rather than raw network reachability.

7. Re-run the corrected command and confirm that a real payload is saved normally.

   $ wget --debug --output-file="$HOME/wget-debug/wget-ok.log" https://httpbin.org/bytes/128 -O debug-ok.bin
   $ ls -lh debug-ok.bin
   -rw-r--r--  1 user  user   128B Mar 27 06:55 debug-ok.bin

   Keep the success test small so the second trace stays easy to compare with the failing run.

   Related: How to set connection and read timeouts in wget

8. Remove or rotate the debug logs once the root cause is documented.

   $ rm -f "$HOME/wget-debug/wget-403.log" "$HOME/wget-debug/wget-ok.log"

   Leaving verbose debug logs on disk increases the chance of exposing internal URLs, tokens, or request metadata later.