How to automatically follow HTTP redirects with cURL

An HTTP redirect is a response from a server that instructs the client to access a different URL. Common types of HTTP redirects include the 301 (permanent) and 302 (temporary) status codes. These are used when content is moved or URL structures are updated. However, client-side redirects using HTML meta tags or JavaScript are not supported by cURL, which only handles HTTP redirects.

By default, cURL does not follow HTTP redirects automatically. It will display the initial redirect response and stop there. For tasks involving automated requests, such as accessing APIs or scraping websites with redirected content, following these redirects is necessary to reach the final destination.

To configure cURL to follow redirects automatically, specific options must be used in the command. You can also adjust the maximum number of redirects or set this behavior globally in the .curlrc file. These options ensure cURL behaves in line with your expectations when handling redirects.

Steps to configure cURL for following HTTP redirects:

Open your terminal. - Run cURL against a URL that issues a redirect.

$ curl -i http://example.com
HTTP/1.1 301 Moved Permanently
Location: http://www.example.com

Use the -L option to follow the redirect.

$ curl -L http://example.com
<!DOCTYPE html>
<html>
  <head><title>Example Domain</title></head>
  <body>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents.</p>
  </body>
</html>

The -L option tells cURL to follow redirects. It will automatically follow all 3xx responses, including 301 and 302 redirects.

Set the maximum number of redirects to follow with –max-redirs if necessary.
```
$ curl -L --max-redirs 3 http://example.com
```
Be cautious when increasing the redirect limit to avoid potential infinite loops. The default limit is 50.
Add the location directive to the .curlrc file to follow redirects automatically for all requests.
```
$ echo 'location' >> ~/.curlrc
```
Set the global maximum number of redirects in the .curlrc file.
```
$ echo 'max-redirs = 10' >> ~/.curlrc
```
Test the configuration by running cURL on a URL that redirects multiple times.
```
$ curl -L http://example.com/multiple-redirects
```
Verify that cURL follows all redirects correctly and reaches the final destination URL.

Author: Mohd Shakir Zakaria
Mohd Shakir Zakaria is an experienced cloud architect with a strong development and open-source advocacy background. He boasts multiple certifications in AWS, Red Hat, VMware, ITIL, and Linux, underscoring his expertise in cloud architecture and system administration.

Discuss the article:

Comment anonymously. Login not required.