Questions about brute force attacks on the admin username, coming from amazon IP addresses

I’ve noticed this once myself (can’t remember if it was the same plugin). The captcha sat there, but just submitting the form didn’t trigger an error and worked perfectly, logging me in.

My #1 advise: use a plugin to rename wp-login.php to something else. It will effectively stop these bots (and delay an attacker that is specifically targeting you, but those are very rare), and you just tell your legitimate users about the new URL to use for login. Obviously, that won’t be an option if you have thousands of users, but for your average company site, it is.

You might also want to look into disabling XMLRPC and the REST API if you don’t use them, as they provide more attack surface.

Other than that, it sounds like you’re already set up quite well, and an active stance on security is always a great starting point.