Add robots.txt to root

First of all, in order for WordPress to generate a robots.txt for you you must be using a non-default permalink structure. Make sure you’ve selected an option in the Settings > Permalinks menu.

Also, if a robots.txt file exists at your root directory it will override the setting in WordPress. It looks like you already have a robots.txt file and that is the reason the wordpress setting is ignored.

Leave a Comment