A sitemap is a file, most often in XML format, that lists the URLs of a website intended to be crawled and indexed by search engines. This file acts as a structured inventory that Google’s or Bing’s bots consult to identify available pages. Understanding how it works helps to grasp why its presence directly influences a site’s visibility in search results.
The role of the lastmod field in the freshness of an XML sitemap
Most guides present the sitemap as a simple list of URLs. In practice, the value of the sitemap mainly depends on the reliability of its metadata, particularly the lastmod field.
This field indicates to the bots the date of the last significant modification of a page. When this date reflects a real content change, search engines prioritize the recrawl of the concerned page. When it is artificially updated, without any real modification, Google eventually ignores the signal.
Since the removal of the sitemap ping interface by Google, the Hi Business sitemap illustrates an approach where the file is generated with reliable modification dates, which remains the recommended method for signaling fresh content to search engines.
The official documentation from Google Search Central, updated in 2026, emphasizes this point: a sitemap whose lastmod is consistently reliable carries more weight in crawl decisions than a file submitted manually repeatedly.
XML Sitemap and crawl budget: an underestimated technical lever
Crawling bots have a limited amount of time to scan each site. This time, often referred to as crawl budget, varies depending on the size of the site, its popularity, and the quality of its internal architecture.
A well-constructed XML sitemap directs bots to the important pages. Conversely, a sitemap that includes redirected URLs, 404 error pages, or content blocked by the robots.txt file wastes this budget on unnecessary resources.
What the sitemap should not contain
- URLs returning a 3xx code (redirects), which force the bot to follow an additional chain before reaching the final page
- Pages marked as noindex, as they are not intended to appear in search results
- Canonicalized URLs pointing to another page, as they create redundancy that the engine must resolve itself
- Orphan pages without real content, which dilute the value of the signals sent to the bots
An effective sitemap contains only indexable and useful URLs. Each unnecessary entry reduces the share of the crawl budget allocated to strategic pages. On an e-commerce site with several thousand product listings, this optimization makes a real difference in the speed of indexing new content.
Technical limits of the sitemap file and managing large sites
The XML format imposes strict constraints. A single sitemap file cannot exceed 50,000 URLs or weigh more than 50 MB uncompressed. Beyond this, the sitemap must be split into several files and grouped into a sitemap index.
This sitemap index is itself an XML file that points to each sub-sitemap. Google, Bing, and other engines read it as a single entry point. Submission can be done through three channels: the Google Search Console, the Search Console API, or a Sitemap directive in the robots.txt file.
Specialized sitemaps for rich content
Sites that host images, videos, or news articles can use specialized sitemaps. These files use dedicated tags to describe the type of content, its caption, duration, or publication date.
A video sitemap, for example, provides the title, description, and thumbnail of each piece of content. This metadata facilitates appearance in Google’s rich results, where videos receive distinct display with a visual preview. For news sites, the news sitemap signals articles published in the last 48 hours, which accelerates their consideration by Google News.
XML Sitemap and internal linking: two complementary tools, not interchangeable
A common mistake is to treat the sitemap as a substitute for internal linking. Both serve distinct functions.
Internal linking structures navigation for visitors and distributes authority among pages. The sitemap, on the other hand, serves exclusively the bots. A sitemap never compensates for a failing internal linking structure. If a strategic page is not accessible by any internal link, search engines will consider it secondary, even if it appears in the sitemap.
The reverse scenario also exists: a well-linked page but absent from the sitemap will likely be discovered by the bots through links. The sitemap accelerates this discovery, especially for recent content or deep architectures where some pages are several clicks away from the homepage.
Auditing your sitemap with Google Search Console
The Search Console provides a report dedicated to submitted sitemaps. This report displays the number of detected URLs, the number of URLs actually indexed, and any errors encountered during crawling.
The gap between submitted URLs and indexed URLs is a key indicator. A significant discrepancy signals a quality issue in the sitemap content: error pages, duplicates, or content deemed too weak by the engine.
- Regularly compare the number of URLs in the sitemap with the number of indexed pages in the coverage report
- Check that new pages appear in the sitemap within hours of their publication
- Remove from the sitemap any URL that returns an HTTP status different from 200
Auditing the sitemap via Search Console often reveals otherwise invisible indexing issues. This regular check allows for correcting anomalies before they affect the ranking of pages in search results.
The XML sitemap is not a file that is generated once and forgotten. Its maintenance, the reliability of its metadata, and the strict filtering of the URLs it contains determine its real usefulness for SEO. A clean, updated file that is properly submitted remains one of the few technical levers whose effect on crawling is directly measurable in the Search Console.