How to solve Robots.txt problem with WordPress fresh install?

Your robots file as of right now:

User-agent: Googlebot
Disallow: 
User-agent: *
Disallow: 
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Sitemap: http://www.yadfaeq.com

You aren’t specifying anything for Googlebot so you should remove it. Additionally, Disallow: could be interpreted as Disallow: / which would block your entire site.

What your robots file should be:

User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Sitemap: http://www.yadfaeq.com

Google often caches the robots.txt file so it may take some time for it to check for a new one. I suggest you use Google Webmaster Tools to help control what Google is indexing.

I do not see the changes you said you made to your robots.txt file. Perhaps your server is caching it as well?

Related Posts:

  1. robots.txt Not Updating
  2. How to remove WordPress Virtual Robots.txt that keeps overriding my root robots.txt? [closed]
  3. Setting up robots.txt with Yoast’s Seo plugin
  4. Robots.txt not updating
  5. NoFollow Entire Website
  6. add post content in meta description in yoast
  7. Disallow author with robots.txt
  8. Set different Canonical url programmatically for every post
  9. Yoast SEO: title tags are repeating my website name and slogan [closed]
  10. How to ensure that a landing page created with WordPress does not enter the sitemap?
  11. SEO optimization gone not so well
  12. ASCII titles with Yoast SEO plugin
  13. Embed sitemap in a page
  14. charset problem in wordpress because of nginx config
  15. How do I create custom tags for WordPress SEO? [closed]
  16. Yoast SEO plugin – Sitemap links greyed out & page priority
  17. How to dissallow all wordpress posts in robots.txt?
  18. Robots.txt file not updating
  19. Facing Problem with Description in Post and Home Page in WordPress
  20. Tagline showing instead of meta description!
  21. Yoasts plugin sitemap not detecting correct category URL format
  22. How Do I Set the Page Title Dynamically?
  23. Is sanitize_title enough to generate post slugs?
  24. Creating 301 Redirects for Post, Page, Category and Image URLs?
  25. How do I set up a custom URL shortener for my posts?
  26. Does the ‘WordPress Order’ feature, within the WordPress Dashboard, have any impact on site architecture?
  27. Inserting Post Using wp_insert_post. How to Fill Yoast Plugin SEO Fields
  28. Migrating from other CMS to WP – losing SEO juice?
  29. Blog not indexed [closed]
  30. Remove date from Google Search Results Description
  31. Duplicate Australian E-Commerce site
  32. Add image tags to native WordPress XML Sitemaps
  33. What does short_ping do?
  34. Can I change the contents of a description meta tag generated by Yoast?
  35. Hard code a domain into Yoast SEO canonical URLs
  36. Change Site Title and Description html tags if not home page
  37. What does it mean by “Home page / Archives” in JetPack Site Stats? [closed]
  38. WordPress Blog and SEO
  39. Will using REST API impact SEO? [closed]
  40. Enabling compression in .htaccess file is causing 500 internal error
  41. is it good to use WordPress in the same url extension in terms of seo?
  42. Overwrite YoastSEO meta-tags with another page’s [closed]
  43. Getting error when trying to implement site-wide breadcrumbs in Twentytwentyone Theme [closed]
  44. Why does Google Image Search still display my images?
  45. Yoast SEO plugin “could detect no serious SEO problems” on dashboard
  46. Reactjs headless WordPress theme configure SEO
  47. How to Change The Title in WordPress For SEO
  48. install robots.txt to wordpress website admin
  49. Search Console detects Category Pagination data as Duplicate Meta Description
  50. WordPress thinks my custom taxonomy pages are search pages #seo
  51. What is/are the best thing(s) I can do to make my WordPress blog more popular? [closed]
  52. how to set title of each page or post for SEO
  53. Show Sitename on Yoast SEO Title tag [closed]
  54. Retrieve a value from Yoast SEO to use to set a default twitter card image honoring overrides
  55. Yoast SEO settings unavailable in an archive page from wordpress plugin
  56. How to remove Date from Meta Description on SERP?
  57. how to 301 redirect from a plugin folder
  58. Is Shared server effect SEO
  59. Yoast taking over my WordPress title tag [closed]
  60. Yoast primary category query modification
  61. WordPress yoast seo plugin – parse snippet var in meta description
  62. How to stop showing my under development site on Google Search [closed]
  63. Search results show a different site [closed]
  64. Fix description of website on Google Search result
  65. SEO is not working on pages [closed]
  66. Google isn’t able to search my website
  67. Pinging or sitemap plugin: which practice is better
  68. Does having two installations of WordPress negatively affect SEO? [closed]
  69. Maintain SEO of a website while migrating it to WordPress?
  70. Word Press Permalink Change
  71. I have redesigned a website and my blog page has different permalinks
  72. custom wordpress theme and yoast seo injecting text in body
  73. WordPress site for increase SEO ranking in other e-commerce store
  74. Remove Noindex?
  75. Is it possible to change the permalink structure without changing the old permalinks to posts and without 301 redirects?
  76. WordPress site using Yoast SEO being blocked from indexing by robots.txt
  77. Permalink not changing according to post type
  78. Show Yoast SEO meta tags in Custom WordPress Templates (Theme)
  79. Sitemap not containing all pages
  80. Metatag Discription on site [closed]
  81. Not sure if to keep this or not (noindex question)
  82. Yoast Seo Plugin meta title not showoing [closed]
  83. My blog name refer to different domain in SERP
  84. Why I can not modify the appereance of my title web site in google? [closed]
  85. Sub category text in the URL of wordpress site
  86. Best Way To Structure 3 Tiered Site
  87. deeplink juggernaut – remove settings
  88. How to change the post title to h1 [closed]
  89. Not Being Indexed by Search Engines [closed]
  90. Disallow /cgi-bin or not? [closed]
  91. Dispise all efforts, open graph is a total mess
  92. Google Tag Manage – to use or not to use
  93. Possible to change meta data single quotes to dobule quotes in yoast wordpress seo? [closed]
  94. Post title not output correctly
  95. How to Make your Category Titles have H1 tags?
  96. Can I delete the redirection in WordPress after switching URLs manually?
  97. seo meta description issue in google search
  98. If a landing page is my home page, and it shouldn’t contain navigation links, then how should it lead to other pages? [closed]
  99. How to improve WordPress website SEO and traffic, and or fix SEO issues
  100. How to fix Links do not have descriptive text PageSpeed Insights
techhipbettruvabetnorabahisbahis forumutaraftarium24eduedueduedusedusedueduedueduedus