It depends which function you are using for querying those categories, but all of them has hide_empty
argument
for example for:
<?php
get_terms( $taxonomies, $args );
$taxonomies = array(
'post_tag',
'my_tax',
);
$args = array(
'hide_empty' => true,
);
?>
by default, hide_empty argument is empty so check if it’s false in you code.