This is exactly how wordpress works.
For each Custom Post type Create an archive-custom_post_type_name.php and a single-custom_post_type_name.php
WordPress automatically loads that file when someone goes to the url.
so if your custom posts types were koolio and notsokool then you would need these 4 files in your theme child (you’re using a child theme, right?!)
archive-koolio.php
single-koolio.php
archive-notsokool.php
single-koolio.php
I would take the single file out of your theme folder and copy it to your child them, then add the CPT extension. Do the same for the archive template.
rememeber you’ll have to flush the re-write rules if your plugin isn’t set to do this by itself…just go to seetings/permalinks and hit save.