Before publishing a Shopify theme change, save the store’s live /robots.txt file and compare it with the proposed output. Check it again on the public domain after release. A theme can look correct while serving different crawl rules, and a copied robots.txt.liquid template does not necessarily produce the same file as Shopify’s current generated default.
This check is for a merchant or release owner changing themes, uploading a theme package or reviewing an old customization. You need the current file, the candidate theme and a short list of URLs whose intended crawl behavior is known. You do not need to remove every block reported by Search Console.
Separate the theme file from the file crawlers receive
Shopify supplies a generated robots.txt file for stores. A custom templates/robots.txt.liquid file replaces that generated output. Liquid is Shopify’s template language: the code in the theme is rendered into the text a crawler receives at /robots.txt. Review that output, not just whether a file with the right name exists.
Shopify recommends using its Liquid objects when customizing rules instead of freezing the whole file as plain text. There is an important qualification: the default groups exposed to a custom template can differ from the platform’s generated file. Shopify currently identifies /search and /policies/ as examples of older blocks that can remain in a custom template’s output even though the generated default no longer includes them.
- Shopify: generated robots output and custom templates
- Shopify: current defaults and older template rules
That does not mean every store should delete its template. First establish why it exists. A deliberate crawler restriction or extra sitemap entry may still be required. Restoring defaults without reviewing those choices can remove useful work just as copying an old file can preserve an obsolete rule.
Record how the new theme reached the store
The transfer route matters. Shopify documents that uploading a theme through the Themes section of the admin does not import robots.txt.liquid. Its guidance says ThemeKit or command-line changes preserve the file. Do not apply that statement to every possible deployment script: check what your actual release includes and what the target theme contains.
For a same-theme version update, use the customization review in our theme-update guide, then add this separate crawl check. For a replacement theme, inspect both themes explicitly. Record the published theme name, candidate name, release time and person who can restore the previous configuration. If nobody knows why a rule was added, mark it for review instead of silently dropping it.
Compare one public URL through the change
Consider an illustrative shipping-policy page at /policies/shipping-policy. The store wants Google to be able to read it. Before a theme switch, the generated file has no /policies/ block. The replacement theme contains an older custom template that renders Disallow: /policies/ for the relevant crawler. The page still opens for shoppers, but that crawler’s access has changed.
The useful comparison is the rule’s effect on the same URL. A text difference alone can include harmless formatting changes. Conversely, an unchanged theme file can render different defaults over time. Keep the saved output with its date so your team can explain what the crawler was actually offered.
Ask the developer to identify the applicable user-agent group and matching rules before changing a line. User-agent identifies the crawler addressed by a group. Removing one visible Disallow line without checking other matching rules can leave the behavior unchanged or affect more URLs than intended. Review a narrow correction against the desired crawl behavior.
Use a small release check with meaningful examples
- ✓Save the current public /robots.txt output, its domain and the time checked. Save the existing custom template too, if present.
- ✓Inspect the candidate theme for templates/robots.txt.liquid and identify each intentional customization. Compare its rendered output with the saved live file.
- ✓Choose a homepage, a priority product, a primary collection and any URL affected by a custom rule. Write down whether each should be crawlable and why.
- ✓Include a control URL that should remain blocked under the store’s policy, such as a cart route. Check that a correction has not opened unrelated paths.
- ✓After publishing, repeat the checks on the ordinary public domain. If host-specific rules exist for Markets domains, check each affected host. Save the result and release owner.
If the candidate’s rendered output cannot be verified before release, record that gap and arrange an immediate live check with a recovery owner. A theme preview screenshot cannot answer this question. Keep the prior template and output available; after any restoration, verify the live response again rather than assuming the rollback worked.
Keep crawl access separate from search visibility
A robots.txt block asks a crawler not to fetch a URL. It does not reliably remove that URL from Google’s results. Google can discover an address through other links. If removal is the actual goal, noindex needs to be visible to Google when it crawls the page; blocking that crawl can prevent Google from seeing the instruction. Google does not support a Noindex line inside robots.txt.
After a correction, use Search Console’s URL Inspection to check the affected live URL and compare it with the earlier report. Search data can lag behind a release. Shopify says “Indexed, though blocked” is often expected for intentionally blocked pages, so the report alone is not a reason to erase platform defaults. A passing live check establishes current behavior, not a promised indexing date or ranking improvement.
Recent X discussion about Shopify’s robots file and AI checkout is another reason to keep the scope precise. Those posts are a discovery signal, not proof of how a particular store behaves. This article checks open-web crawl rules around a theme release; it does not infer agentic-checkout availability or Shopify Catalog distribution from those rules.
Questions before changing the file
Does robots.txt protect confidential pages?
Will restoring Shopify defaults be enough?
Do I need a storefront redesign or SEO app for this check?
Before your next theme release, add /robots.txt to the evidence you save alongside storefront checks. Start with one important URL and one intentional block. Inficial can help trace a mismatch to the theme customization and verify the smallest suitable correction.
Sources
