function noindex_comments_pages() { global $cpage; if (!empty($cpage) && $cpage > 1) { echo ‘<meta name=”robots” content=”noindex”>’; echo “\n”; } } add_action( ‘wp_head’, ‘noindex_comments_pages’, 9 );
WordPress responds to requests to a robots.txt with dynamic content if such a file does not exist. That’s one way how the settings from wp-admin/options-privacy.php are used. I recommend to create a static robots.txt, just to make sure no plugin is getting in your way. Sample robots.txt User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes … Read more
If you use Yoast SEO, you can go to Users > All Users, and in the Yoast SEO settings, check Do not allow search engines to show this author’s archives in search results. See this link. OR With robots.txt, you can add: User-agent: * Disallow: /author/* OR You can also add the robots meta-tag to … Read more
Well, that message is pretty self explanatory. There is no sitemap linked in your robots.txt file. So you check if you have a sitemap (if not install a plugin that will generate one for you) and add this line to your robots.txt file in the root of your site: Sitemap: http://www.example.com/sitemap.xml
Two options: Create a static file robots.txt. Highly recommended. Filter ‘robots_txt’: add_filter( ‘robots_txt’, ‘wpse_77969_robots’ ); function wpse_77969_robots() { status_header( 204 ); return ”; }
Add a simple checkbox to the post editor to toggle the generation of a meta field that disallows search engines to index your content. Then hook into wp_head, check for the setting and print that field. Sample code for a plugin: add_action( ‘post_submitbox_misc_actions’, ‘show_noindex_checkbox’ ); add_action( ‘save_post’, ‘save_noindex’, 10, 2 ); add_action( ‘wp_head’, ‘insert_noindex_meta’ ); … Read more
Despite the fact your site lacks a robots.txt file, it might be worth making one to allow for Googlebot and other bots to crawl your website. Please use this example: User-agent: * Allow: / Disallow: /wp-content Note: Please remove or amend the /wp-content disallow on the robots.txt, if for whatever reason, you’d like your images … Read more
First of all, in order for WordPress to generate a robots.txt for you you must be using a non-default permalink structure. Make sure you’ve selected an option in the Settings > Permalinks menu. Also, if a robots.txt file exists at your root directory it will override the setting in WordPress. It looks like you already … Read more
How exactly are you setting up your home page? I think the problem is with it having unwanted pagination in first place and not that pagination being indexed. In general robots.txt file is good way to prevent indexing in bulk. I think it would be following directive in your case (please test it so it … Read more
Similar problem occurred to me, and I got the solution following these: Step 1: Take a backup of your .htaccess file and then remove it (Don’t worry, on next refresh WordPress will create one for you) Step 2: If there’s no robots.txt exists, create one with blank page Step 3: Resubmit the sitemap to google … Read more