Is there a way to visualize / print out which templates were used to render a given piece of UI?

Here is something i use when developing themes:

add_filter( 'template_include', 'store_template_file', 999 );
function store_template_file( $template ){
    $GLOBALS['current_theme_file'] = basename($template);
    return $template;
}

add_action('wp_footer','print_theme_file');
function print_theme_file(){
    if( !isset( $GLOBALS['current_theme_file'] ) || is_admin() )
        return;
    echo '<!-- Current theme file: ' .  $GLOBALS['current_theme_file'] . '-->';
}

Once you paste that in your theme’s functions.php file it will print out an HTML comment at the footer with the theme file’s name.