If you are trying to get the number of terms your post have, then you should use wp_get_post_terms()
.
$sizeCount = count(wp_get_post_terms($post->ID, 'sizes'));
$colorCount = count(wp_get_post_terms($post->ID, 'colors'));
echo $sizeCount . ' ' . $colorCount;