Blocking admin-ajax.php from outside domain

If the point of the file is to allow the website to asynchronously
post data why would it even be allowed to be accessed directly,
especially from an ip that is not the ip of the site itself?

AJAX requests come from the user’s IP address, not the site’s, because an AJAX request is by definition a request made by the user’s browser.

Would I break things if I blocked all ips that are not my site ip from
POSTing to admin-ajax.php?

Yes, you would break all AJAX requests that use admin-ajax.php, including those used by WordPress core.

I have reCaptcha installed but they are getting past it.

If reCAPTCHA is implemented correctly these direct requests should fail because the server should be validating the CAPTCHA on the server. If you’ve implemented reCAPTCHA yourself you need to make sure this server-side validation is occurring. If you didn’t, then you will need to raise this issue with the developer of the plugin you’re using.