Faceted Navigation SEO: Stop Filter URLs Eating Your Crawl

Why one category page turns into thousands of URLs

Filters, sort orders, view toggles and tracking parameters each create a URL that looks new to a crawler. A crawler cannot tell a URL is useless until it has fetched it, so Google can end up crawling plenty of low-value pages before it reaches the ones you care about. Google also notes that this can use up real server and rendering resources, and leaves less crawl time for new, useful pages. Here is what it looks like on a shoe store:

/shoes/ /shoes/?color=black /shoes/?color=black&size=9&sort=price-asc /shoes/?sort=price-asc&size=9&color=black

The last two URLs show the same products. Only the order of the parameters changed, and a crawler treats them as two different pages.

Decide first: which filtered pages should be allowed to rank

Before touching any code, sort your URLs into buckets. The last column is our editorial call, not a Google rule, so check it against your own Search Console data.

Route one: stop Google crawling filters you do not need

Google's documentation splits the fix in two. If you do not need filtered URLs in Search, you can prevent crawling. There are two ways to do it:

• robots.txt disallow. Block the parameter-based filter URLs, and keep the unfiltered listing page and the individual product pages crawlable.

• URL fragments. Put the filters after a # instead of in query parameters. Google generally ignores fragments for crawling and indexing.

A starting point for the robots.txt route. Test every pattern before you ship it, because one greedy rule can block pages you want:

User-agent: * Disallow: /*?*sort= Disallow: /*?*size= Disallow: /*?*color=

Google also warns that canonical and nofollow are generally less effective in the long run than robots.txt blocking or fragments. If crawl waste is your main problem, this route is the stronger one.

Route two: keep filtered pages crawlable and point them at a clean URL

Use this route for the filtered pages you want indexed. Google's list of ways to make crawling more efficient:

• Point filtered URLs at a less-filtered version with rel="canonical". This may reduce crawling of the non-canonical versions over time.

• Add rel="nofollow" to links pointing at filtered pages. It only works if every link to that URL carries it.

• Use the standard & to separate parameters. Commas, semicolons and brackets are often not recognised as separators.

• If filters sit in the URL path, keep their order fixed and block duplicate filters.

• Return a 404 for empty or nonsensical combinations and for pagination pages that do not exist. Do not redirect them to a generic not-found page.

One trap: do not stack both routes on the same URL. A page blocked in robots.txt cannot be fetched, so Google never sees the canonical tag on it. Google also says robots.txt is not a canonicalization tool, since a blocked URL can still be indexed without its content.

Canonical tags: the rules that actually matter

A canonical is a signal, not a command. Google's page calls these methods signals and preferences, and says that if you specify nothing it picks the best version itself. So treat the tag as a strong hint and keep everything else on your site pointing the same way. Google ranks the methods like this:

Backing up the URL you prefer The tag itself is one line in the head:

The rules from Google's guidance:

• Use an absolute URL, not a relative path.

• Give each page one canonical. Do not send different ones through different methods.

• Add a self-referencing canonical on the canonical page itself.

• Put it in the head, and use either the link element or the HTTP header, not both.

• Never use a URL fragment as the canonical.

• If JavaScript sets it, put it in the HTML source if you can, and do not let scripts change an existing canonical element.

• Link internally to the canonical URL every time, and keep your sitemap to canonical URLs.

And three things Google says not to use for canonicalization: robots.txt, the URL removal tool, and noindex. Noindex removes the page from Search entirely, which is not what you want for a duplicate whose signals you hoped to merge into another URL.

A 30-minute audit you can run this week

• Open the page indexing report in Search Console and look for URLs that contain a ? or an &.

• Crawl the site with any crawler and export every URL with a parameter. Count them against your real product count.

• Check your server logs for how often Googlebot requests parameter URLs compared with product pages.

• Open three filtered URLs, view source and read the canonical tag. Is it absolute, single and pointing where you expect?

• Load a filter combination that cannot return products. Does it give a real 404 status?

• Check that internal links and the XML sitemap only use clean, canonical URLs.

On Shopify, check what canonical your theme prints on filtered collection URLs rather than assuming it is right. For more on the store side, see our guides on Shopify development in Delhi NCR, AI-ready product feeds and AEO-ready website architecture.

FAQ

Should I add noindex to filtered pages?

Not as a way to handle duplicates. Google says noindex is not recommended for canonicalization because it removes the page from Search entirely. If you want the signals merged into a cleaner URL, use rel="canonical". If you never want the filtered URLs crawled at all, use the robots.txt or fragment route.

Is rel=canonical a command Google has to follow?

No. Google describes canonical annotations as signals. A rel="canonical" tag and a 301 redirect are both strong signals, a sitemap entry is a weak one, and Google can still pick a different URL if your signals disagree.

Should I use robots.txt or canonicals for filter URLs?

It depends on whether you need those URLs in Search. If you do not, block them or move the filters behind a #. If you do, keep them crawlable and use canonicals. Google adds that the first approach tends to work better in the long run.

What should a filter combination with no results return?

A 404 status on the requested URL. Google says not to redirect these to a generic not-found page.

Does Google crawl anything after the # in a URL?

Generally no. Google ignores URL fragments for crawling and indexing, which is why filters placed after a # do not create new crawlable pages.

Sources:

• Google: Managing crawling of faceted navigation URLs

• Google: Consolidate duplicate URLs and specify a canonical