That’s not how switch_to_blog works.
When you load WP, it loads the theme and plugins into the global namespace, with no sandboxing or walls. When you then call switch_to_blog, there’s no way to put all of that to one side and load the plugins/theme from the other blog.
What switch_to_blog will do though is switch the database used and the data context for stuff. It just won’t reload the theme/plugins, and since post types and taxonomies are defined in code, and not the database, they won’t change.
This means only the theme and plugins of the site the request was made to will be loaded, you can’t call functions or reference post types that only exist on the other blog. As a result get_post_types will always return the same thing by design.
The only reliable way you have of querying which post types another site has, or running code in that sites context with all the plugins, is to make a HTTP request, or spawn a WP CLI process from PHP.
Eitherway this approach is a dead end, not worth pursuing, and rests upon a flawed assumption of how WP loads code