Get all sub-categories of a parent category

In reply to your updated code, you should know that the $post_terms = wp_get_object_terms() would only return categories that directly attached to the post, so I’d just use wp_list_categories() to get and display the child categories for the given (or a known) parent category.

And here’s the code that works well for me which displays up to three levels of child categories including those having no posts. This code would replace the lines starting from the $page_id = get_queried_object_id(); to the $terms = wp_list_categories(); as in the question, and I’m using get_ancestors() instead of (your) smart_category_top_parent_id():

$page_id = get_queried_object_id();

$category = get_the_category( $page_id );
$catid = ( ! empty( $category ) ) ? $category[0]->cat_ID : 0;

if ( $catid ) {
    // Get all parent IDs.
    $parents = get_ancestors( $catid, 'category', 'taxonomy' );

    $top_level_cat = ( count( $parents ) > 1 ) ?
        $parents[ count( $parents ) - 2 ] : // use the second parent category ID, or
        array_pop( $parents );              // otherwise, we'll use the first one

    $terms = wp_list_categories( array(
        'title_li'   => '',
        'echo'       => false,
        'taxonomy'   => 'category',
        'show_count' => 1,
        'child_of'   => $top_level_cat,
        'style'      => 'list',
        'hide_empty' => 0, // include empty categories
        'depth'      => 3, // and up to 3 levels depth
    ) );
}