Adding all custom capabilities to admin

What is the correct approach here? Should all custom caps be added to
admin role manually like they are added to other roles?

Yes, because register_post_type() will not automatically assign the capabilities to any users or user roles, including administrators, but if Multisite is enabled, then those who are super admins would always be able to manage (e.g. edit or delete) the posts in the post type, unless if do_not_allow is present in their capabilities (as returned by map_meta_cap()) — see WP_User::has_cap().