Count tags for a specific category

count how many posts are tagged but only include those posts that
belong to a specific category

You can’t do that via get_terms(), but you can make a WP_Query query and use the $found_posts property to get the total posts.

So for example, in your foreach, you can use the tag and cat parameters like so — just change the 1,23 with the correct category ID(s):

$query = new WP_Query( array(
    'tag'            => $t->slug,
    'cat'            => '1,23', // category ID(s)
    'fields'         => 'ids',
    'posts_per_page' => 1,
) );

$count = $query->found_posts; // the total posts

Note: I used 'fields' => 'ids' and 'posts_per_page' => 1 because I just wanted to know the total posts, hence I don’t need the full post data and I also set the posts per page to only 1.

And please check the documentation for more information on the parameters and other details.