How to create an XML sitemap index

Large sites often publish separate sitemap files for product catalogs, articles, media, or other URL sets that change on different schedules. A sitemap index gives crawlers one stable entry point without merging those inventories into one oversized file.

Each sitemap entry contains an absolute loc value for a child sitemap that is already published on the same site. Publishing the index at the site root gives crawlers a stable URL while the child sitemap files may remain elsewhere on that site. On a Debian or Ubuntu web server, the private working copy stays outside the document root until the privileged install step publishes it.

A sitemap index can contain up to 50,000 child sitemap locations and be no larger than 50 MB uncompressed. The optional lastmod value describes when the child sitemap file changed, not when any page inside it changed, so omit the value when the generator cannot keep it accurate.

1. Create a private sitemap index working directory outside the site's public document root.

   $ mkdir --parents "$HOME/sitemap-index-work"

2. Enter the private sitemap index working directory.

   $ cd "$HOME/sitemap-index-work"

3. Create sitemap-index.xml with one absolute loc value for each published child sitemap.

   <?xml version="1.0" encoding="UTF-8"?>
   <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <sitemap>
       <loc>https://www.example.com/sitemaps/products.xml.gz</loc>
     </sitemap>
     <sitemap>
       <loc>https://www.example.com/sitemaps/articles.xml.gz</loc>
     </sitemap>
   </sitemapindex>

   Every child sitemap must remain on the same site as the index, and special characters in its URL require XML escaping. The optional lastmod belongs only when it records the child file's actual modification time.

4. Download the official sitemap index schema into the private working directory.

   $ curl --fail --silent --show-error --output siteindex.xsd https://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd

5. Validate the private sitemap-index.xml file against the official schema before deployment.

   $ xmllint --noout --schema siteindex.xsd sitemap-index.xml
   sitemap-index.xml validates

6. Remove the temporary sitemap index schema after validation.

   $ rm -- siteindex.xsd

7. Set DOCUMENT_ROOT to the active virtual host's document root.

   $ DOCUMENT_ROOT=/var/www/html

   /var/www/html is a common Apache and Nginx default on Debian and Ubuntu; the active virtual host configuration is authoritative.

8. Deploy the validated index from the private working directory to the selected document root.

   The sudo install command is the apply boundary because it replaces the live root-level file; an incorrect document root publishes the index on the wrong virtual host.

   $ sudo install --mode=0644 sitemap-index.xml "$DOCUMENT_ROOT/sitemap-index.xml"

9. Fetch the public sitemap index URL to confirm it returns the expected sitemapindex document.

   $ curl --fail --silent --show-error https://www.example.com/sitemap-index.xml
   <?xml version="1.0" encoding="UTF-8"?>
   <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <sitemap>
       <loc>https://www.example.com/sitemaps/products.xml.gz</loc>
     </sitemap>
     <sitemap>
       <loc>https://www.example.com/sitemaps/articles.xml.gz</loc>
     </sitemap>
   </sitemapindex>

   A redirect, login page, HTML error response, or XML document without the expected child loc values means the public handoff is not complete.