Query only current post type using taxonomy

If you look at WordPress’ available query variables, you will notice post_type. You will need to add that to your URL:

$terms = get_the_terms( $post->ID , 'category' ); 
foreach ( $terms as $term ) {
  $term_link = get_term_link( $term, 'category' );
  if( is_wp_error( $term_link ) ) 
  continue;
  $term_link = add_query_arg(
    array(
      'post_type' => $post->post_type
    ),
    $term_link
  );
  echo '<a href="' . $term_link . '">' . $term->name . '</a>';
} 

Reference:
https://developer.wordpress.org/reference/functions/add_query_arg/