Ok editing the default-ssl.conf as described here (or in my case /etc/apache2/sites-enabled/mysite.com-le-ssl.conf) to add auth settings to the end of the Virtual Host block works fine now.
So it’s now:
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName mysite.com
ServerAlias www.mysite.com
ServerAdmin webmaster@localhost
DocumentRoot /var/www/mysite.com
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/mysite.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mysite.com/privkey.pem
<Directory "/var/www/mysite.com">
AuthType Basic
AuthName "Restricted Area"
AuthUserFile /etc/wordpress/.htpasswd
require valid-user
</Directory>
</VirtualHost>
</IfModule>
The page also explains how to use .htaccess files by modifying AllowOverride.
I should probably comment on the guide I originally followed