As long as a plugin is blocking excess login attempts, then I am not sure that additional protection is needed. The plugin should be protecting against the attempts at credential guessing.
There will always be login attempts on sites, even if they aren’t WP sites. You could look at your access logs (filtering by a 40x result code) and see tons of attempts to access things that aren’t available on your site. Part of the way things are.
As long as you are blocking excess logins, and have strong passwords for your accounts, I don’t see the advantage of an additional layer of security (via htaccess passwords).
Those login attempts will always be happening. Your defense against them is the plugin that limits login attempts, and strong passwords.