Instead of get_the_terms
try using get_terms
and set the hierarchal argument to true. Also make sure you show empty so you are not hiding your empty categories.
Try something like this:
$args = array(
'hide_empty' => false,
'hierarchical' => true
);
$terms = get_terms('portfolio_pt', $args);
foreach($terms as $term)
echo $term->name;