Here is one of the possible solutions. There are some other recursive solutions for multi child levels. According to your question which is a one child level.
The quick test proved to work which is conducted with default themes and default taxonomy Category.
// first get the parent only $terms
// because of unknown sorting
$parent_terms = [];
$terms = get_terms( array(
'taxonomy' => 'category', // change to your taxonomy name
'hide_empty' => false,
) );
foreach( $terms as $term ) {
if( $term->parent == 0 ) {
$parent_terms[] = $term;
}
};
// create level render
foreach( $parent_terms as $parent ) {
$child_terms = get_terms( array(
'taxonomy' => 'category',
'child_of' => $parent->term_id,
'hide_empty' => false,
) );
echo '<h4>' . $parent->name . '</h4>';
echo '<ul>';
foreach( $child_terms as $child ) {
echo '<li>' . $child->name . '</li>';
}
unset($child_terms); // avoid garbage for reuse
echo '</ul>';
}