Thanks to @jacob peattie for helping me to figure this out. The issue is that I had WP_CONTENT_URL set incorrectly. Removing it fixed the issue. I could have added /wp-content to the url and it would’ve solved the issue as well.
To explain why this caused in issue, I turned to the docs. This page explains that the WP_CONTENT_URL variable (which I understand is set to $WP_SITE_URL . '/wp-content') which the content_url function returns (after removing any extra trailing slashes). This function is used by both WordPress core as well as any extra plugins to the relative url to where 3rd party content (which includes themes and plugins, as well as uploaded media) is stored.
Because the stylesheets, javascript, fonts, etc were stored with the theme, the browser was unable to find them as it was told to look in the wrong spot. The correct use to change WP_CONTENT_URL would be is you serve the wp-content url from another domain( possibly through a CDN) that is seperate from the rest of your wordpress site.