How to retry transient errors with cURL

Transient network failures such as dropped TCP connections, DNS timeouts, and overloaded gateways often introduce flakiness into automation, CI pipelines, and cron jobs that depend on web APIs. Allowing curl to automatically retry these temporary faults increases resilience and reduces the need for manual reruns when endpoints are healthy but momentarily unreachable.

Retry behavior in curl is controlled by options such as --retry, --retry-delay, --retry-max-time, and additional switches like --retry-connrefused and --retry-all-errors. With these options enabled, curl repeats failed transfers for conditions considered transient, including selected HTTP 5xx responses and many connection-level errors, while still respecting per-attempt timeouts and the overall retry budget.

Retry configuration benefits most from conservative defaults tuned for idempotent HTTP GET and HEAD requests, especially on shells based on Ubuntu and other Linux distributions. Excessively high retry counts or enabling --retry-all-errors against non-idempotent POST or PATCH endpoints risks duplicated writes, extra load on degraded services, and extended job runtimes, so each call pattern deserves explicit tuning.

1. Confirm the installed curl version in a terminal.

   $ curl --version
   curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9
   Release-Date: 2022-01-05
   Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps
   Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM SSL TLS-SRP
   ##### snipped #####

   Retry-related switches such as --retry, --retry-delay, --retry-max-time, --retry-connrefused, and --retry-all-errors can be listed explicitly with curl --help all | grep retry on modern builds.

2. Send an idempotent HTTP GET request with a bounded retry count to cover common transient failures.

   $ curl --retry 5 https://example.com/api/health
   {"status":"ok","uptime":7}

   Without an explicit delay, curl applies an internal increasing backoff between retry attempts for conditions treated as transient, which reduces hammering an unstable endpoint.

3. Apply an explicit retry delay and an overall retry window to keep waiting time under control during outages.

   $ curl --retry 5 --retry-delay 3 --retry-max-time 30 https://example.com/api/health
   {"status":"ok","uptime":19}

   Large values for --retry, --retry-delay, or --retry-max-time can cause long-running jobs that repeatedly hit degraded services, amplifying load during incidents and delaying failure visibility.

4. Extend retry coverage to connection failures and refused connections only for safe, idempotent operations.

   $ curl --retry 5 --retry-delay 2 --retry-max-time 30 --retry-connrefused --retry-all-errors https://example.com/api/health
   {"status":"ok","uptime":33}

   Enabling --retry-all-errors on requests that change state, such as POST or PATCH calls, can duplicate record creation, trigger repeated side effects, or cause extra charges if an upstream processes a request multiple times.

5. Inspect retry behavior and final status by printing connection statistics while discarding the response body.

   $ curl --retry 5 --retry-delay 1 --retry-max-time 15 --write-out 'num_connects=%{num_connects}\nexit_code=%{exitcode}\n' --output /dev/null --silent https://example.com/api/health
   num_connects=2
   exit_code=0

   Success indicators include a shell exit code of 0 and a num_connects value greater than 1, which together confirm that curl retried at least once and ultimately completed a successful transfer.

6. Capture a reusable retry configuration for scripts by wrapping common options in a shell variable or alias.

   $ CURL_RETRY_OPTS='--retry 5 --retry-delay 2 --retry-max-time 30'
   $ curl $CURL_RETRY_OPTS https://example.com/api/health
   {"status":"ok","uptime":58}

   Packaging retry settings into a variable or alias keeps command lines readable and ensures consistent behavior across cron jobs, CI stages, and interactive troubleshooting sessions.