Template hierarchy about pagination after front-page.php

Add a filter to frontpage_template and check the paged query var, adding your own template when it’s beyond the first page. I use paged.php in this example:

function wpa56889_template_filter( $templates="" ){
    $paged = get_query_var( 'paged' );
    if( $paged > 1 ) :
        if( !is_array( $templates ) && !empty( $templates ) ) :
            $templates = locate_template( array( "paged.php", $templates ), false );
        elseif( empty( $templates ) ) :
            $templates = locate_template( "paged.php", false );
        else :
            $new_template = locate_template( array( "paged.php" ) );
            if( !empty( $new_template ) ) array_unshift( $templates, $new_template );
        endif;
    endif;
    return $templates;
}
add_filter( 'frontpage_template', 'wpa56889_template_filter' );

Note the Filter Hierarchy Codex entry incorrectly lists the filter as front_page_template, but the function that applies this filter appears to sanitize the underscore from the variable before calling it, making it just frontpage_template.