how to display child terms with parent terms in custom taxonomy?

Have just tweaked your code to achieve what you need. See, if this works for you –

$taxonomyName = "age";
$parent_terms = get_terms($taxonomyName, array('parent' => 0, 'orderby' => 'slug', 'hide_empty' => false));   
echo '<ul>';
foreach ($parent_terms as $pterm) {
    $terms = get_terms($taxonomyName, array('parent' => $pterm->term_id, 'orderby' => 'slug', 'hide_empty' => false));
    foreach ($terms as $term) {
        echo '<li>'.$pterm->name.', <a href="' . get_term_link( $term->name, $taxonomyName ) . '">' . $term->name . '</a></li>';  
    }
}
echo '</ul>';

Specifically, I have added $pterm->name within the <li> tag which should give you parent term name along with each of its child term.

Please note: I have not tested it.