First of all, in order for WordPress to generate a robots.txt for you you must be using a non-default permalink structure. Make sure you’ve selected an option in the Settings > Permalinks menu.
Also, if a robots.txt file exists at your root directory it will override the setting in WordPress. It looks like you already have a robots.txt file and that is the reason the wordpress setting is ignored.
Related Posts:
- ssl_error_rx_record_too_long and Apache SSL [closed]
- Environment variable to control java.io.tmpdir?
- What is spark.driver.maxResultSize?
- Using wp-uploads instead of wp-content/uploads
- Cannot get pages or posts to display
- get_pages and number – no output; no errors;
- how do I disable new user registration email for specific admin user?
- Disable WordPress Redirect From terminal
- Move wordpress files to sub-folder
- www.website.com works but website.com redirects to website.com/index.html
- Copy wordpress settings to another blog
- Where do I add my own configuration needs and how do I access them?
- how to server over http and https with one installation
- command for checking Apache configuration
- Not able to access wordpress website publicly installed on azure VM having apache server as xampp
- Reading settings from app.config or web.config in .NET
- How to import a module given the full path?
- Encrypt Password in Configuration Files?
- Encrypt Password in Configuration Files?
- How to solve the error “SCRIPT7002: XMLHttpRequest: Network Error 0x80070005, Access is denied.” in IE
- How to set NotebookApp.iopub_data_rate_limit and others NotebookApp settings in JupyterHub?
- How to intercept already localized scripts
- Control verbosity level of WP DEBUG?
- How can I debug the TTFB in WP?
- Multisite Network Port Num Issues?
- Improve wordpress security by hiding non public resources
- Individual Widgets per Page
- What is a good robots.txt?
- Robots.txt not updating
- NoFollow Entire Website
- How do I transition multiple installations to a single Multisite installation?
- WordPress Multisite with multiple domains using Nginx
- Hosting multiple WordPress sites on single server – best practices?
- Getting trackback spam, even with trackbacks disabled
- How to use live images on local install?
- Stuck on my server root folder, robots.txt file not deleting
- Generate WordPress salt
- Multisite behind Varnish
- WordPress PHP error handling and reporting in production environment
- How to combine javascript files without plugins?
- Where are a theme’s options stored and can I export them to be imported into freshly installed themes?
- Generate WP-CLI @alias for each site on multisite
- Declaratively Configuring WordPress in XML or JSON or YAML
- Correct way to hide pseudo pages from being shown?
- Override Current Theme Setting in wp_config.php
- References to other site everywhere in WordPress site
- What are best practices for configuring a server for WordPress sites?
- WordPress salts set in config and database
- WordPress behind Proxy – Mixed Content
- Create unique robots.txt for every site on multisite-installation
- Generating robots.txt dynamically
- Making a configurable field translatable
- Steps to debug on a blank screened wordpress install
- Unwanted “crawl delay: 10” line added to my robots.txt
- What is the best way to provide plugin users with a way to customized the styles
- Why does Google Webmaster Tools say my whole site is blocked by robots.txt when I have no robots.txt?
- Repeatable configuration package
- Preventing Search Engines Indexing Pages 2, 3 and More?
- Broken image multisite
- Google saying is blocked by robots.txt without robots.txt on my website
- Hide pages from google and visitors
- While Using Static Pages, How Can I Get /blog into the URL of Each Post?
- Hide root site in Multisite install
- Set wp-content folder to Dropbox folder
- How to backup and restore configurations
- How to change how long items are kept in the trash?
- Allow download_url for lan addresses
- Subdomain and subdirectories together in one installation
- What should be disabled to run on an internal network with no access to the Internet?
- How to stop WordPress showing a virtual Robots.txt?
- “No sitemap linked in your robots.txt file”
- Can I configure WordPress to use postfix without a plugin?
- No configuration file found and no installation code available. Exiting
- Where should I add the necessary code to add an extra attribute to the kses allowed tags array?
- Alias domain to Multi-site installation in Apache
- How can I make wordpress suppress mysql errors?
- Does WordPress MultiSite use separate MySql databases?
- Putting main WordPress blog under a different folder name
- Per site configuration
- Disallow author with robots.txt
- .htaccess file missing from WordPress install directory
- Set WP_MAX_MEMORY_LIMIT higher than PHP.ini memory_limit
- Error establishing a database connection on Installation
- Permalinks not working
- WordPress with Lan and Wan Access?
- Worried I have a funky .htaccess for WP site in light of strange search engine behavior
- Can somebody tell me how I am supposed to be using blogs.dir for network / MU sites?
- Set noindex page-comment from Pages 2, 3 and More?
- Why edits to wp-config.php must come before “That’s all” comment
- nginx.conf appeared in all WordPress installations on folder
- How do I hide out of stock products from homepage but not from catalog?
- wp-config dynamic hostname in WP_HOME and WP_SITEURL
- robots.txt Not Updating
- Where is the “Privacy” menu for the “WP Robots Txt” plugin?
- How to solve Robots.txt problem with WordPress fresh install?
- Questions about Robots.txt
- Gutenberg Block Development: Trying to add custom js script to npm start command by modifying webpack.config
- WordPress Ignoring .user.ini
- Why does Google Image Search still display my images?
- Configure WordPress Installation autoinstalls via Softaculous in WHCMS [closed]