Every year, competent teams launch sites carrying the staging robots.txt or a site-wide noindex tag. Both are invisible in the browser and both stop the site being indexed. Two checks, ten minutes after launch.
Both of these are trivial. Both cost weeks of traffic. Both happen to teams who know better, because neither is visible in a browser.
1. The staging robots.txt went live
Staging environments are usually blocked with a Disallow: / for all user agents. When the site is deployed, that file is deployed with it, and the new site politely asks every search engine not to crawl it.
Check: open yourdomain.com/robots.txt in a browser within ten minutes of launch. If it contains Disallow: / under User-agent: *, fix it now. Then confirm it again the following morning, because a scheduled deployment can reintroduce it.
2. A site-wide noindex is still switched on
In WordPress this is a single checkbox — Settings, Reading, Discourage search engines. In most frameworks it is a meta tag or an X-Robots-Tag header set by an environment variable. Both survive a launch happily.
Check: view the source of the home page and a product or service page, and search for noindex. Also check the HTTP response headers, because an X-Robots-Tag: noindex header does not appear in the HTML at all and is the harder of the two to spot.
The rest of the ten-minute list
- Canonical tags point at the live domain, not staging.
- The sitemap is present, lists live URLs and has been submitted in Search Console.
- HTTPS and the www or non-www choice resolve consistently, with one canonical version and redirects for the others.
- Analytics and conversion tracking fire on the live domain — verified with a real test transaction, not a debug view.
- Twenty redirects tested by hand, including the strangest URLs on the old site.
The habit worth building
Set a calendar reminder for 24 hours after every launch, with these six checks in the body. Search Console will eventually tell you about a blocked site, and by then you will have lost a fortnight of crawling you did not need to lose.
Looking for more? Browse all resources.