Why an SEO Tool Might Report a Different Status Code Than Your Browser

You open a URL in a browser and see the page load fine — a 200. A tool reports the same URL as a 301, a 404, or something else entirely. Neither is necessarily malfunctioning. A URL doesn’t have one fixed status code; it has whatever status code the server decides to return to a given request, and a browser and an automated tool don’t always send the same request.

The request isn’t as identical as it looks

Both requests target the same URL, but everything else about them can differ, and servers are free to respond to any of it.

The Accept-Language header often differs. Your browser sends your configured language preference; an automated tool may send none, or a default. A server doing language negotiation — the mechanism covered in Vary, content negotiation, and one URL serving two pages — can legitimately serve a different representation, or redirect to a language-specific path, based on that header alone.

Cookies and session state differ. Your browser carries whatever cookies a prior visit set; a fresh automated request carries none. A server that redirects first-time visitors to an onboarding path, or serves a paywall-lite version to unauthenticated requests, produces a different response to a client with no session than to your logged-in browser.

The user agent itself can change server logic, separate from any bot-detection system — a server that serves a lightweight page to clients it detects as bots, or that has different caching rules keyed on user agent, produces different output for that reason alone.

Geographic routing differs by request origin. A tool running from cloud infrastructure in one region and your browser on a residential connection in another can be routed to different edge nodes or different geo-targeted content, each with its own status behavior.

Cache state differs by cache key. If the response varies and the Vary header is set correctly, each client gets the representation that matches its own headers. If Vary is missing where it’s needed, whichever client’s response got cached first is served to everyone else regardless of their own headers — which means the “wrong” status code you’re seeing might be a stale cached response served to a client whose real request would return something else.

Redirect-following differs too

An automated tool’s HTTP client makes its own decision about whether to follow a redirect automatically and how many hops to accept, covered in what a redirect chain looks like from a scraping API’s perspective. A browser follows redirects transparently and shows you the final page; a tool configured to report the first response rather than follow the chain will report the 301 your browser never shows you, because your browser resolved it before you looked.

Two tools can also disagree with each other, for the same reasons

The discrepancy isn’t limited to “tool versus browser.” Two automated tools checking the same URL can report different status codes for identical reasons — different declared user agents, different cookie handling (some tools maintain a session across a crawl, others fetch each URL cold), and different redirect-following configuration. A status-code audit that pulls from two different tools and treats a mismatch as one of them being wrong is asking the wrong question; the useful question is which request each tool actually made, which requires checking each tool’s documented fetch behavior rather than assuming both sent the same request to begin with.

This is also why a single tool’s historical report can stop matching a later report from the same tool: if the vendor changes its declared user agent, adds cookie persistence, or changes its default redirect-following limit between versions, the same URL can start returning a different status without anything on the site changing at all.

What to check, in order

Confirm what was actually served, not what you assume was served, using the log-reading approach in reading log files for crawl evidence — the log records the real status code for the real request, independent of either the browser’s or the tool’s report.

Reproduce the tool’s request, not your own. curl with an empty cookie jar and the tool’s declared user agent (if documented) gets you closer to what it actually saw than opening the URL in a browser that already has a session and a language preference set:

curl -sI -A "toolname/1.0" https://example.com/some-path

Check whether Vary is set, and on what. If the response depends on a header and Vary doesn’t declare it, the two clients aren’t disagreeing about the URL — a cache is serving one of them a stale representation.

Check for a redirect the tool stopped at and your browser walked through. A single curl -I (not -IL) shows the first hop only, which is often exactly what a status-code discrepancy is: the tool reporting hop one, your browser having silently followed to hop three.

What you can and can’t conclude

You can conclude that both reports may be accurate for the request each client actually made. You can’t conclude that either one is “the” status code for the URL — a URL fronted by content negotiation, session-aware routing, or an uncorrected Vary gap doesn’t have a single status code independent of who’s asking.

The durable point

A status code is the server’s answer to a specific request, not a fixed property of a URL. Two clients that appear to be asking the same question can be sending meaningfully different requests, and the discrepancy is usually explained by finding which header, cookie, or redirect-following behavior differed — not by assuming one report is simply wrong.