|
By default, Magento often adds in a robot.txt file onto your site to prevent search engines from indexing your site. This is done so that when you are developing a site (and don`t have real prices or products), your site won`t show up in Google`s search results.
We`ve had several cases where this is set incorrectly in the Magento Admin after the site is launched. It`s a really minor thing, but to a search engine like Bing, Google or Yahoo, this file basically tells search engines to go away and not pay attention to your store.
For example, your settings are most likely setup incorrectly if your meta information on your site contains:
meta name=”robots” content=”*”
ormeta name=”robots” content=”NOINDEX,NOFOLLOW”
INSTEAD, it should read something more like this:
meta name=”robots” content=”INDEX,FOLLOW”
… so that search engines will continue to index your site.
To check or update your Magento store`s Robot settings you can go to your Magento Admin Panel, then to System -> Configuration. On the left side click on the Design tab and scroll down until you see the “Default Robots” setting. Make sure your live or production site is set to “INDEX, FOLLOW” while any of your development or testing sites are set to “NOINDEX, NOFOLLOW”.
|
