Disallow /cgi-bin or not? [closed]

It can’t hurt (except your robots.txt is publicly accessible, anyone can see your directory), but bots who honor the robots.txt file, will skip that directory anyway. If you don’t use any CGI scripts, I would simply block it in .htaccess.

Related Posts:

  1. How Do I Set the Page Title Dynamically?
  2. Is sanitize_title enough to generate post slugs?
  3. Robots.txt not updating
  4. Adding rel=”next” & rel=”prev” for paginated archives
  5. NoFollow Entire Website
  6. Creating 301 Redirects for Post, Page, Category and Image URLs?
  7. How do I set up a custom URL shortener for my posts?
  8. Does the ‘WordPress Order’ feature, within the WordPress Dashboard, have any impact on site architecture?
  9. SEO – Two WordPress websites on same domain name
  10. add post content in meta description in yoast
  11. Migrating from other CMS to WP – losing SEO juice?
  12. Blog not indexed [closed]
  13. Remove date from Google Search Results Description
  14. Duplicate Australian E-Commerce site
  15. Add image tags to native WordPress XML Sitemaps
  16. What does short_ping do?
  17. Disallow author with robots.txt
  18. Seo WordPress Pages on google [closed]
  19. Hard code a domain into Yoast SEO canonical URLs
  20. How to remove duplicate blog title
  21. Change Site Title and Description html tags if not home page
  22. Set different Canonical url programmatically for every post
  23. What does it mean by “Home page / Archives” in JetPack Site Stats? [closed]
  24. WordPress Blog and SEO
  25. Will using REST API impact SEO? [closed]
  26. How to Appease the Google PageSpeed Gods with regards to image optimization?
  27. Enabling compression in .htaccess file is causing 500 internal error
  28. robots.txt Not Updating
  29. is it good to use WordPress in the same url extension in terms of seo?
  30. How to solve Robots.txt problem with WordPress fresh install?
  31. Yoast SEO: title tags are repeating my website name and slogan [closed]
  32. How to ensure that a landing page created with WordPress does not enter the sitemap?
  33. How may I see all the pages on my wordpress website and see when they were created?
  34. Search Console detects Category Pagination data as Duplicate Meta Description
  35. SEO optimization gone not so well
  36. Automatic SEO for wordpress [closed]
  37. How to hide sensitve page from google spiders and non-membres in wordpress!
  38. Custom Canonical URLs
  39. ASCII titles with Yoast SEO plugin
  40. What is/are the best thing(s) I can do to make my WordPress blog more popular? [closed]
  41. how to set title of each page or post for SEO
  42. Search Engine Visibility checkbox after indexed
  43. how to 301 redirect from a plugin folder
  44. best way to stop this wordpress duplicate content
  45. Embed sitemap in a page
  46. charset problem in wordpress because of nginx config
  47. Is Shared server effect SEO
  48. wp_strip_all_tags [closed]
  49. How to remove WordPress Virtual Robots.txt that keeps overriding my root robots.txt? [closed]
  50. Updating old website links to new website links [closed]
  51. Search results show a different site [closed]
  52. Fix description of website on Google Search result
  53. SEO is not working on pages [closed]
  54. Google isn’t able to search my website
  55. Pinging or sitemap plugin: which practice is better
  56. How do I create custom tags for WordPress SEO? [closed]
  57. The inner mechanism of WP SEO plugins
  58. Does having two installations of WordPress negatively affect SEO? [closed]
  59. Maintain SEO of a website while migrating it to WordPress?
  60. Word Press Permalink Change
  61. WordPress Meta Title Separator
  62. Yoast SEO plugin – Sitemap links greyed out & page priority
  63. Customizing SERP content for WordPress Blog
  64. Pages and posts with Rel Canonical issues across entire site
  65. Setting up robots.txt with Yoast’s Seo plugin
  66. How to dissallow all wordpress posts in robots.txt?
  67. I have redesigned a website and my blog page has different permalinks
  68. WordPress site for increase SEO ranking in other e-commerce store
  69. Remove Noindex?
  70. WordPress Embed Page no-index meta data missing
  71. Unlimited false pages showing in Sitemap?
  72. Sitemap not containing all pages
  73. How to add blog in default WordPress post type for SEO standard rule?
  74. Metatag Discription on site [closed]
  75. Not sure if to keep this or not (noindex question)
  76. Facing Problem with Description in Post and Home Page in WordPress
  77. My blog name refer to different domain in SERP
  78. mobile usability issues with /wp-content/uploads/ folders
  79. Sub category text in the URL of wordpress site
  80. Change WordPress url for newly formed posts or pages
  81. OpenGraph descriptions not showing when using WordPress SEO (Yoast)
  82. Best Way To Structure 3 Tiered Site
  83. deeplink juggernaut – remove settings
  84. How to change the post title to h1 [closed]
  85. special XML to wordpress posts
  86. wordpress theme: on site seo [closed]
  87. Tagline showing instead of meta description!
  88. Anchors in WordPress not followed/ indexed in sitemap.xml
  89. Not Being Indexed by Search Engines [closed]
  90. Dispise all efforts, open graph is a total mess
  91. Google Tag Manage – to use or not to use
  92. Multiple Domains Same Install Different Static Pages
  93. Post title not output correctly
  94. How to Make your Category Titles have H1 tags?
  95. Can I delete the redirection in WordPress after switching URLs manually?
  96. Hide my staging subdomains from search results without impacting the visibility of the root domain
  97. If a landing page is my home page, and it shouldn’t contain navigation links, then how should it lead to other pages? [closed]
  98. How to improve WordPress website SEO and traffic, and or fix SEO issues
  99. Yoasts plugin sitemap not detecting correct category URL format
  100. How to fix Links do not have descriptive text PageSpeed Insights