The post won’t use those templates because those aren’t intended for posts. They’re intended for the archive pages for those terms/categories/tags.
What’s more, it doesn’t make sense for it to work that way as it introduces ambiguity. Case in point, if I have category-red.php and category-blue.php, which template gets loaded if I visit a post that’s in both red and blue categories?
Instead, take a look at the template hierarchy diagram, you’ll very likely need to implement a check in your posts template yourself. Also consider adding a single.php, and post.php template to your theme