How to configure retries for transient errors with cURL

Transient network failures such as dropped TCP connections, DNS timeouts, and overloaded gateways often introduce flakiness into automation, CI pipelines, and cron jobs that depend on web APIs. Allowing curl to automatically retry these temporary faults increases resilience and reduces the need for manual reruns when endpoints are healthy but momentarily unreachable.

Retry behavior in curl is controlled by options such as --retry, --retry-delay, --retry-max-time, and additional switches like --retry-connrefused and --retry-all-errors. With these options enabled, curl repeats failed transfers for conditions considered transient, including selected HTTP 5xx responses and many connection-level errors, while still respecting per-attempt timeouts and the overall retry budget.

Retry configuration benefits most from conservative defaults tuned for idempotent HTTP GET and HEAD requests, especially on shells based on Ubuntu and other Linux distributions. Excessively high retry counts or enabling --retry-all-errors against non-idempotent POST or PATCH endpoints risks duplicated writes, extra load on degraded services, and extended job runtimes, so each call pattern deserves explicit tuning.

1. Confirm the installed curl version in a terminal.

   $ curl --version
   curl 8.5.0 (aarch64-unknown-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
   Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.6
   Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
   Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

   Retry-related switches such as --retry, --retry-delay, --retry-max-time, --retry-connrefused, and --retry-all-errors can be listed explicitly with curl --help all | grep retry on modern builds.

2. Send an idempotent HTTP GET request with a bounded retry count to cover common transient failures.

   $ curl --silent --show-error --fail --retry 5 http://api.example.net/flaky
   curl: (22) The requested URL returned error: 503
   curl: (22) The requested URL returned error: 503
   {
     "status": "ok"
   }

   Without an explicit delay, curl applies an internal increasing backoff between retry attempts for conditions treated as transient, which reduces hammering an unstable endpoint.

3. Apply an explicit retry delay and an overall retry window to keep waiting time under control during outages.

   $ curl --silent --fail --retry 5 --retry-delay 2 --retry-max-time 10 http://api.example.net/flaky
   {
     "status": "ok"
   }

   Large values for --retry, --retry-delay, or --retry-max-time can cause long-running jobs that repeatedly hit degraded services, amplifying load during incidents and delaying failure visibility.

4. Extend retry coverage to connection failures and refused connections only for safe, idempotent operations.

   $ curl --silent --show-error --retry 3 --retry-delay 1 --retry-max-time 5 --retry-connrefused --retry-all-errors http://api.example.net:81/
   curl: (7) Failed to connect to api.example.net port 81 after 0 ms: Couldn't connect to server
   curl: (7) Failed to connect to api.example.net port 81 after 0 ms: Couldn't connect to server
   curl: (7) Failed to connect to api.example.net port 81 after 0 ms: Couldn't connect to server
   curl: (7) Failed to connect to api.example.net port 81 after 0 ms: Couldn't connect to server

   Enabling --retry-all-errors on requests that change state, such as POST or PATCH calls, can duplicate record creation, trigger repeated side effects, or cause extra charges if an upstream processes a request multiple times.

5. Inspect retry behavior and final status by printing the last HTTP code and exit status while discarding the response body.

   $ curl --silent --fail --retry 5 --retry-delay 1 --retry-max-time 10 --write-out 'http_code=%{http_code}\nexit_code=%{exitcode}\n' --output /tmp/flaky-output http://api.example.net/flaky
   http_code=200
   exit_code=0

   Success indicators include a shell exit code of 0 and a final http_code that matches the expected status from the endpoint after retries succeed.

6. Capture a reusable retry configuration for scripts by wrapping common options in a shell variable or alias.

   $ CURL_RETRY_OPTS='--fail --retry 5 --retry-delay 2 --retry-max-time 30'
   $ curl --silent $CURL_RETRY_OPTS http://api.example.net/flaky
   {
     "status": "ok"
   }

   Packaging retry settings into a variable or alias keeps command lines readable and ensures consistent behavior across cron jobs, CI stages, and interactive troubleshooting sessions.