AWS WordPress Install – pages/posts 404 on restart until updating permalinks

This answer was based off the comment from @Rup on the question. I needed to add a .htaccess file to my source. I did this with the default contents:

# BEGIN WordPress

RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

# END WordPress

which I got from here: https://wordpress.org/support/article/htaccess/

Adding this at the root of my deploy package (uploaded source) and the problem went away.