URL Depth and Click Depth Are Different Things

example.com/men/shirts/pink and example.com/mens-pink-shirts can be equally easy or equally hard for a crawler to reach. The number of slashes in a path is a naming convention. What costs a crawler something is click depth: how many links it has to follow from a page it already knows about to arrive at this one.

The two get conflated because they usually correlate. Sites that nest their URLs deeply also tend to bury the corresponding pages behind several navigation steps, so the deep-looking URL really is the hard-to-reach page. The correlation is real and the causation runs through the link graph, not through the string.

Why the string doesn’t cost anything

A crawler’s frontier is a set of URLs discovered from links, sitemaps and other references. To fetch /a/b/c/d/e/, it needs one thing: a reference to that URL. Path segments aren’t traversed — there’s no requirement to know /a/b/ exists before fetching /a/b/c/, and directory levels aren’t opened in sequence. One link from your homepage makes a five-segment URL exactly one click deep.

Two caveats, both about humans rather than crawlers. Path segments are readable, and a URL that describes its own position is easier to reason about, easier to redirect in bulk, and easier to match with a robots or CDN rule. And a path that encodes a taxonomy encodes a decision about that taxonomy: /men/shirts/pink commits you to colour living under type living under department, which is fine until pink shirts also need to live under a seasonal collection. That’s an information-architecture cost, not a crawl cost.

What click depth costs

Click depth from a well-linked entry point does have observable consequences:

• Discovery latency. A page four clicks from anything a crawler visits often gets found later.
• Recrawl frequency. Pages that sit at the periphery of the internal link graph tend to be revisited less often, so changes to them take longer to be reflected.
• Signal distribution. Internal links are how authority moves around a site. A page reachable only through a chain of low-value intermediates receives whatever survives that chain. Internal Linking as Plumbing covers the mechanism.

None of that is a threshold. There’s no depth at which pages stop being indexed. It’s a gradient, and its practical effect is largest on big sites where the tail is genuinely deep and the crawl doesn’t reach all of it in a useful timeframe.

Measuring the one that matters

Depth is a property of your link graph, so measure it there. Any crawler that records the level at which each URL was first discovered will give you a distribution:

1. Crawl from the homepage with JavaScript rendering matching what you actually ship.
2. Export URL and discovery depth.
3. Bucket it: how many URLs at depth 1, 2, 3, 4, 5+.
4. Cross-reference the deep buckets against URLs that earn impressions or revenue.

The output that matters is the list of pages you care about sitting at depth four or more. That’s a work list. A histogram with a long tail of pages nobody needs indexed is not a problem to solve.

Two measurement traps. First, if your navigation is built client-side, a crawler that doesn’t execute it sees a much deeper site than your rendered one — see Navigation That Only Exists After JavaScript Runs. Crawl both ways and compare; the difference is the part of your link graph that depends on rendering. Second, a sitemap makes URLs discoverable without making them shallow. Listing every URL in a sitemap fixes discovery and does nothing for the signal distribution, which is why What an XML Sitemap Is Actually For is a narrower tool than it looks.

Where depth actually comes from

Almost always one of three places:

Pagination. A category with 40 pages of results puts the last item dozens of clicks deep if the only path to it is next-next-next. This is the biggest single source of depth on catalogue sites, and it’s structural — no amount of URL flattening touches it. Pagination Without rel=next and rel=prev covers the options.

Faceted navigation. Filter combinations generate a large, shallow-looking space that consumes crawl capacity while the pages you care about wait behind it. The problem there is breadth rather than depth, and the fix is limiting what’s crawlable: Faceted Navigation and the Crawl Space It Opens.

Orphaned or near-orphaned pages. Pages reachable only from a sitemap, an old email, or one link in a three-year-old post. Depth is effectively infinite for the ones with no internal link at all, and the fix is a link from somewhere relevant, not a URL rewrite.

Why URL flattening disappoints

Rewriting /men/shirts/pink to /mens-pink-shirts changes every URL in the section. That means redirects for all of them, a window where both forms exist, internal links to update everywhere, and permanent redirect infrastructure to maintain. What it does not do is change how many clicks it takes to reach the page — the navigation is identical afterwards.

If the pages were slow to be found and slow to be recrawled, they will still be slow, because the cause was the click path. You’ll have spent a migration to change a string.

The version of this change that does pay off is the one where the URL rewrite is incidental to a navigation change: you add the pages to a hub that’s linked from the header, or you cross-link siblings, or you replace next-page-only pagination with something that reaches the tail in two clicks. Then depth genuinely drops — and it would have dropped without touching a single URL.

The test before you commit: if you did nothing to the URLs and only changed the linking, would the problem you’re trying to solve be solved? If yes, do that, and leave the paths alone. If no, the problem was never depth.