Compare Crawls: Snapshots and Diff to Track SEO Progress Over Time

The Compare crawls page lets you create snapshots of your crawl data at a point in time and compare two snapshots to see what changed: new URLs, removed URLs, HTTP status changes, and issue counts. It’s the place to prove that your fixes and content work actually moved the needle and to catch regressions before they hurt traffic.

Sections and how to use them

Crawl snapshots

• Snapshot list — Shows all snapshots for the property. Each snapshot has a label (e.g. “Pre-migration”, “After meta fix”) and usually a created date. Use labels that describe the state of the site (e.g. “Before redirect cleanup”).
• Create snapshot — You enter an optional label and click Create snapshot. The app captures the current crawl data (all stored URLs, statuses, issue counts, etc.) into a new snapshot. Create one after each major change (e.g. after fixing redirects, after adding 50 new pages) so you have clear “before” and “after” points.
• Loading — While creating, the button may show a spinner. After creation, the list refreshes and the new snapshot appears.

How to use: Run a crawl, create a snapshot (e.g. “Baseline March”). Make fixes. Re-crawl. Create another snapshot (e.g. “After fixes March”). Now you can compare them.

Compare two snapshots

• Select snapshot A and B — Two dropdowns (or selectors) let you pick the first and second snapshot (e.g. “Baseline” vs “After fixes”). Order can matter for “added” vs “removed” (typically A = older, B = newer).
• Compare — Click Compare to run the diff. The app computes differences between the two snapshots and shows the result.

Comparison result

After comparing, you typically see:

• Summary — Counts such as: URLs only in A, URLs only in B, URLs in both; or “added”, “removed”, “unchanged”. Sometimes total URLs and total issues per snapshot.
• Added URLs — URLs that appear in snapshot B but not in A (new or newly crawled pages). Use this to confirm new content was discovered and to check that new pages don’t have critical issues.
• Removed URLs — URLs that were in A but not in B (deleted, moved, or no longer in crawl). Use this to catch accidental removals or redirects that dropped URLs from the crawl.
• Changed (when available) — Rows where the same URL exists in both but something changed: e.g. HTTP status (200 → 404, or 404 → 301), issue count (increased or decreased). This is where you see “this page was fixed” or “this page got worse”.
• Pagination — If the diff is large, use pagination to move through added/removed/changed lists.
• Open in app — For each URL you can often open it in the page viewer to see current verification details.

How to use: After a round of fixes, compare “before” vs “after”. You want to see: fewer non-200s, fewer pages with issues, and no unexpected removals of important URLs. If issue count went up on some URLs, open them and fix. Use the comparison in reports or standups to show progress.

How Compare crawls helps traffic and SEO

• Prove impact — Show stakeholders that after your work, non-200 count dropped, issue count dropped, or new pages were added. Use numbers and “added/removed/changed” lists as evidence.
• Catch regressions — If you see new broken URLs or status changes from 200 to 4xx/5xx, fix them before they hurt rankings and traffic.
• Track over time — Keep a series of snapshots (e.g. monthly) and compare consecutive pairs to see ongoing improvement or new problems.
• Migration and launches — Before/after migration or big launches, compare snapshots to ensure nothing critical was lost and new URLs are healthy.

Use Compare crawls after every significant crawl and after major SEO or site changes. Combine with the Dashboard (for current metrics), Verifications (to fix the pages that changed), and Action plan (for the next set of tasks) to close the loop: fix → re-crawl → snapshot → compare → report.