Display empty custom taxonomies differently than with one with posts

The number of posts attached can be found here: $term->count.

So you could do the following:

$class_empty_term = '';
if( $term->count == 0 )
    $class_empty_term = 'empty';
$term_list .= '<div class="' . $class_empty_term . ' grid__item ...">';
$term_list .= '<div class="panel text--center">';