How to configure retries for transient errors with cURL

Transient HTTP 503s, short-lived rate limits, and service startup races can make healthy automation look broken when cURL gives up on the first failed transfer. A retry policy keeps API checks, CI jobs, scheduled fetches, and shell wrappers moving through brief instability without turning every temporary fault into a hard failure.

The core retry switch is --retry, which replays transfers when cURL sees timeout conditions or specific transient server responses. Current upstream curl builds also honor Retry-After when a server provides it, while --retry-delay replaces the default exponential backoff, --retry-max-time caps the overall retry budget, and --retry-connrefused expands the retry set to services that have not opened their listening socket yet.

Retries should stay limited to safe, repeatable operations such as GET, HEAD, or requests that are otherwise duplicate-safe. Older curl builds can differ slightly in which HTTP status codes are considered transient by default, and --retry-all-errors is intentionally broad enough to repeat permanent failures too, so it should be reserved for tightly controlled cases instead of added blindly to a global profile.

1. Start a disposable local endpoint that returns 503 twice and then 200 so the retry behavior is visible before applying the same pattern to production URLs.

   $ python3 - <<'PY'
   import http.server
   import socketserver
   
   state = {"hits": 0}
   
   class Handler(http.server.BaseHTTPRequestHandler):
       def do_GET(self):
           if self.path == "/reset":
               state["hits"] = 0
               self.send_response(204)
               self.end_headers()
               return
   
           if self.path == "/flaky":
               state["hits"] += 1
               if state["hits"] < 3:
                   body = f"status=temporarily_unavailable attempt={state['hits']}\n".encode()
                   self.send_response(503)
                   self.send_header("Content-Type", "text/plain; charset=utf-8")
                   self.send_header("Content-Length", str(len(body)))
                   self.send_header("Retry-After", "1")
                   self.end_headers()
                   self.wfile.write(body)
                   return
   
               body = b"status=ok retry_recovered=true\n"
               self.send_response(200)
               self.send_header("Content-Type", "text/plain; charset=utf-8")
               self.send_header("Content-Length", str(len(body)))
               self.end_headers()
               self.wfile.write(body)
               return
   
           if self.path == "/always-503":
               body = b"status=temporarily_unavailable\n"
               self.send_response(503)
               self.send_header("Content-Type", "text/plain; charset=utf-8")
               self.send_header("Content-Length", str(len(body)))
               self.send_header("Retry-After", "1")
               self.end_headers()
               self.wfile.write(body)
               return
   
           self.send_response(404)
           self.end_headers()
   
       def log_message(self, format, *args):
           return
   
   with socketserver.TCPServer(("127.0.0.1", 18081), Handler) as httpd:
       httpd.serve_forever()
   PY

   Leave this process running in its own terminal while the next commands are tested, then stop it with Ctrl+C when the retry checks are finished.

2. Retry a safe GET with a fixed attempt count, explicit delay, and overall retry budget so temporary 503 responses can recover cleanly.

   $ curl --silent http://127.0.0.1:18081/reset -o /dev/null
   $ curl --silent --show-error --fail --retry 2 --retry-delay 1 --retry-max-time 5 --output /dev/null --write-out "response_code=%{response_code}\nexitcode=%{exitcode}\n" http://127.0.0.1:18081/flaky
   curl: (22) The requested URL returned error: 503
   curl: (22) The requested URL returned error: 503
   response_code=200
   exitcode=0

   The repeated 503 lines show the failed attempts, and the final 200 with exit code 0 confirms that the transfer eventually succeeded inside the retry window.

3. Keep the retry window finite so persistent failures stop quickly instead of stretching an outage across an entire job run.

   $ curl --silent --show-error --fail --retry 5 --retry-delay 1 --retry-max-time 2 \
     --output /dev/null \
     --write-out "response_code=%{response_code}\nexitcode=%{exitcode}\ntime_total=%{time_total}\n" \
     http://127.0.0.1:18081/always-503
   curl: (22) The requested URL returned error: 503
   curl: (22) The requested URL returned error: 503
   curl: (22) The requested URL returned error: 503
   response_code=503
   exitcode=22
   time_total=2.018442

   --retry-max-time limits the retry budget, not the runtime of an attempt that is already in progress, so pair it with explicit request timeouts when each transfer also needs a hard ceiling.

4. Add --retry-connrefused when the target service may not have opened its port yet, such as during container startup or rolling restarts.

   $ curl --silent --show-error --fail --retry 2 --retry-delay 1 --retry-max-time 4 --retry-connrefused \
     --output /dev/null \
     --write-out "response_code=%{response_code}\nexitcode=%{exitcode}\ntime_total=%{time_total}\n" \
     http://127.0.0.1:18082/
   curl: (7) Failed to connect to 127.0.0.1 port 18082 after 0 ms: Couldn't connect to server
   curl: (7) Failed to connect to 127.0.0.1 port 18082 after 0 ms: Couldn't connect to server
   curl: (7) Failed to connect to 127.0.0.1 port 18082 after 0 ms: Couldn't connect to server
   response_code=000
   exitcode=7
   time_total=2.029615

   Connection-refused retries are appropriate for short startup races, but they should stay tightly bounded so permanently closed ports or bad addresses are still visible as failures.

5. Reuse the retry policy in a shell array or wrapper so repeated calls stay consistent across scripts, CI stages, and ad-hoc checks.

   $ CURL_RETRY_OPTS=(--silent --show-error --fail --retry 2 --retry-delay 1 --retry-max-time 10 --retry-connrefused)
   $ curl "${CURL_RETRY_OPTS[@]}" https://status-api.example.test/v1/healthz -o /dev/null

   The masked status-api.example.test endpoint shows a typical read-only health check shape. Add --retry-all-errors only when the request is provably duplicate-safe and the input and output paths are not redirected, because that option also retries permanent failures such as 404 responses.