Category count but only if category has at least one post

Have you tried:

wp_count_terms( 'category', array( 'hide_empty' => TRUE ) );

You can check out the source here.