Not sure if you can use a complete different template but you can use different template parts using the global $page variable.
Example:
global $page;
if ( $page == 2 ) {
get_template_part( 'page', 'two' );
} else {
get_template_part( 'page', 'default');
}
This would load the template file page-two.php or page-default.php depending on what page you were on.