How to move wordpress website from hosting account to localhost

The solution was to allow using .htaccess in /etc/apache/apache2.conf by changing from AllowOverride None to AllowOverride All.
More about it here

So that section related to my local website now looks like:

<Directory /var/www/local.webiste.com/public_html>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
        Allow from 127.0.0.1
</Directory>