A production sitemap doubles as a release inventory: each listed URL should still represent the canonical page that visitors and crawlers are meant to reach. Redirects, missing pages, and server failures in that inventory signal drift that can waste crawl activity or hide a broken release path.

A Python standard-library checker can parse one XML urlset and request every loc value without following redirects. Blocking automatic redirect handling keeps the first response visible, so a 301 cannot be mistaken for the 200 returned by its eventual destination.

The input for this method is one public XML urlset rather than a sitemap index, and each canonical URL is expected to return 200 on its first hop. Requests run sequentially with a 15-second timeout; schedule large inventories away from peak traffic because the check sends one GET request to every listed URL.

Related: How to check broken links on your website
Related: How to create an XML sitemap index

Steps to audit website HTTP status codes:

  1. Download the public XML sitemap to sitemap.xml. 
    $ curl --fail --silent --show-error --output sitemap.xml https://www.example.com/sitemap.xml

    The file should contain the canonical public pages being audited.
    Related: How to create an XML sitemap for your website

  2. Create audit_status.py with the imports and redirect-blocking handler.
    audit_status.py
    from collections import Counter
from urllib.error import HTTPError, URLError
from urllib.request import HTTPRedirectHandler, Request, build_opener
from xml.etree import ElementTree
 
 
class NoRedirect(HTTPRedirectHandler):
    def redirect_request(self, request, fp, code, message, headers, new_url):
        return None
  3. Add the XML namespace and sitemap reader below the NoRedirect class in audit_status.py. 
    SITEMAP_NS = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9"}
 
 
def sitemap_urls(path):
    root = ElementTree.parse(path).getroot()
    return [
        node.text.strip()
        for node in root.findall("sm:url/sm:loc", SITEMAP_NS)
        if node.text
    ]
  4. Add the first-hop request and status classification functions below sitemap_urls(). 
    opener = build_opener(NoRedirect())
 
 
def first_hop_status(url):
    request = Request(url, headers={"User-Agent": "HTTP-status-audit/1.0"})
    try:
        with opener.open(request, timeout=15) as response:
            return response.status
    except HTTPError as error:
        return error.code
    except URLError:
        return None
 
 
def status_class(code):
    if code is None:
        return "transport_error"
    if 200 <= code < 300:
        return "success"
    if 300 <= code < 400:
        return "redirect"
    if 400 <= code < 500:
        return "client_error"
    if 500 <= code < 600:
        return "server_error"
    return "other"

    transport_error covers request failures such as DNS errors, TLS validation failures, connection refusals, and timeouts that do not return an HTTP status.

  5. Append the report loop and exit-status check below status_class(). 
    def main():
    urls = sitemap_urls("sitemap.xml")
    if not urls:
        print("ERROR\tno URLs found in sitemap.xml")
        return 2
 
    counts = Counter()
    unexpected = 0
    print("STATUS\tCLASS\tURL")
 
    for url in urls:
        code = first_hop_status(url)
        label = status_class(code)
        counts[label] += 1
        unexpected += code != 200
        print(f"{code if code is not None else 'ERROR'}\t{label}\t{url}")
 
    print(
        "\nSummary: "
        f"total={len(urls)} "
        f"success={counts['success']} "
        f"redirects={counts['redirect']} "
        f"client_errors={counts['client_error']} "
        f"server_errors={counts['server_error']} "
        f"transport_errors={counts['transport_error']}"
    )
    return 1 if unexpected else 0
 
 
if __name__ == "__main__":
    raise SystemExit(main())
  6. Run the completed audit against sitemap.xml. 
    $ python3 audit_status.py
STATUS  CLASS         URL
200     success       https://www.example.com/products/
301     redirect      https://www.example.com/support/
404     client_error  https://www.example.com/retired/
500     server_error  https://www.example.com/checkout/

Summary: total=4 success=1 redirects=1 client_errors=1 server_errors=1 transport_errors=0

    The process exits with status 1 whenever any URL is not exactly 200, so a release job can block on redirects and errors instead of accepting a partial report. A redirect row records only its first response; the HTTP Redirect Checker displays the complete hop chain for that URL.
    Tool: HTTP Redirect Checker

  7. Re-run the audit after the site or sitemap owner resolves every non-200 row. 
    $ python3 audit_status.py
STATUS  CLASS    URL
200     success  https://www.example.com/products/
200     success  https://www.example.com/support/
200     success  https://www.example.com/retired/
200     success  https://www.example.com/checkout/

Summary: total=4 success=4 redirects=0 client_errors=0 server_errors=0 transport_errors=0

    A zero exit status with every row at 200 confirms that the current sitemap inventory has no unexpected first-hop redirect, client error, server error, or transport failure.

Share this guide