Category archive pages do not use archive-{$post_type}.php type templates. Instead they use category-{$category->slug}.php, category-{$category->term_id}.php, category.php, archive.php or index.php depending on the first available template in the hierarchy.
An easy fix would be to just make a copy of archive-{$post_type}.php and renaming it to category.php. WordPress will automatically use this template when youn visit a category page. From what I can gather from your question, you have already use pre_get_posts to add your custom post type to category and tag archives.
The long fix, should you decide that you really really need to use archive-{$post_type}.php, is to make use of the category_template filter to tell WordPress to use this template and not the other templates that might be available in the hierarchy
add_filter( 'category_template', function ( $template )
{
// Try to locate our new desired template
$locate_template = locate_template( 'archive-portfolio.php' );
// If our desired template is not found, bail
if ( !$locate_template )
return $template;
// Our desired template exists, load it
return $locate_template;
});
EXTRA IMPORTANT INFO
If you are in any means using a custom query to include your custom post type into your tag and category archives, remove that custom query and go back to the default loop: (NOTE: The same applies to the home page and index.php)
if ( have_posts() ) {
while ( have_posts() ) {
the_post();
// Add your mark up and template tags
}
}
You should not be seeing any custom post type posts now when you visit a category or tag archive page. To include the custom post type posts, we will use pre_get_posts to add your custom post type to category and tag archive pages
add_action( 'pre_get_posts', function ( $q )
{
if ( !is_admin() // Only target front pages
&& $q->is_main_query() // Only target the main query
&& ( $q->is_category() // Target category archives OR
|| $q->is_tag() // Target tag achives OR
|| $q->is_home() // Target the home page
)
) {
$q->set( 'post_ype', ['post', 'portfolio'] );
}
});