Google is unable to crawl Robots.txt file of my website and please help me
Today I got a mail from Google webmasters, that Google bots are unable to find robots.txt file of my website. This is not actually a problem (from crawling perspective at least, it might be for excluding things from crawling, which the file primarily used for). The robots.txt is neither mandatory on necessary for your site … Read more