Display only children of custom hierachial taxonomy

Based on our comment discussion, you’re using the code in a sidebar widget (that accepts php code) on a single post page.

It ended up being much more complex than expected. This should work though.

<?php
$taxonomy = 'download_category';
$terms    = wp_get_object_terms( get_the_ID(), $taxonomy );
$parents  = array();

// Loop through all album categories.
foreach ( $terms as $term ) {

  // Get the parent terms of each category (if any).
  if ( 0 < $term->parent ) {
    $ancestors = get_ancestors( $term->term_id, $taxonomy );

    // Get the parent term object.
    $parent = get_term_by( 'id', $ancestors[0], $taxonomy );
    $parents[$parent->name] = $parent->term_id;

    // Put children in array to use below.
    $children[] = $term;
  }
}

// Exit if no parents found.
if ( empty( $parents ) ) {
  return _e( 'No parent categories found.' );
}

// Loop through all parents and output their children.
foreach ( $parents as $parent_name => $parent_id ) {

  echo '<h3>' . $parent_name . '</h3>';
  echo '<ul>';
  foreach ( $children as $child ) {
    if ( $child->parent == $parent_id ) {
      echo '<li><a href="' . get_term_link( $child, $taxonomy ) . '">' . $child->name . '</a></li>';
    }
  }
  echo '</ul>';
}
?>