What is an XML Sitemap?

An XML sitemap is a file (usually at yoursite.com/sitemap.xml) that lists URLs you want search engines to know about. It can include optional metadata such as last modified date and priority. Sitemaps don’t guarantee indexing, but they help crawlers discover and prioritise pages, especially on large or complex sites.

When sitemaps help

• Large sites — Thousands of URLs; crawlers may not find everything by following links alone.
• New or updated content — New pages or sections that aren’t well linked yet.
• Deep or parameterised URLs — Pages that are hard to discover through normal navigation.

Structure

• Index sitemap — Points to multiple sitemap files (e.g. by section or type).
• URL sitemap — Contains <url> entries with <loc>, and optionally <lastmod>, <changefreq>, <priority>.

Search engines may ignore priority and changefreq; lastmod can help them decide when to re-crawl.

Common issues

• Missing or broken — Sitemap not found, returns 404, or has invalid XML.
• Out of sync with site — Listed URLs that 404, redirect, or are disallowed by robots.txt.
• Too large — Single sitemap with tens of thousands of URLs; split into multiple sitemaps and use an index.

How BearAudit uses it

BearAudit discovers sitemaps for each property and compares them to what we actually crawl. We report coverage gaps (e.g. URLs in sitemap but not crawled, or crawled but not in sitemap), so you can fix discovery and indexing issues.