I managed to get it working by placing the following in an .htaccess file in my React app folder:
<IfModule mod_rewrite.c>
RewriteEngine on
# Don’t rewrite files or directories
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ – [L]
# Rewrite everything else to index.html to allow html5 state links
RewriteRule ^ index.html [L]
</IfModule>
Also WP custom permalinks work as well.