Why would changing ports from 80 to 8080 cause wordpress to act unexpectedly?

The problem is not in WordPress, it is in your Varnish or Apache configuration.

Static files, such as CSS, JS, Images, these are served by Apache without going through WordPress at all. WordPress will simply use whatever URL you configure it to use. It doesn’t care about port or domains, as such.

So, if your request to your site to get the CSS is returning a 404, why is it doing that? WordPress is not involved, the .htaccess rules WP uses say for Apache to serve any files that exist directly. Take WordPress out of the equation, and just look at why you can’t get the CSS file from that URL.