Get all pages that are published

You can use get_pages to get all the published pages, which is similar to get_posts, as

<?php $args = array(
    'sort_order' => 'asc',
    'sort_column' => 'post_title',
    'hierarchical' => 1,
    'exclude' => '',
    'include' => '',
    'meta_key' => '',
    'meta_value' => '',
    'authors' => '',
    'child_of' => 0,
    'parent' => -1,
    'exclude_tree' => '',
    'number' => '',
    'offset' => 0,
    'post_type' => 'page',
    'post_status' => 'publish'
); 
$pages = get_pages($args); // get all pages based on supplied args

foreach($pages as $page){ // $pages is array of object
   $page_template = get_post_meta($page->ID, '_wp_page_template', true); // Page template stored in "_wp_page_template"

   echo $page_template;
}

?>

Checkout these links for more detail:

get pages: https://codex.wordpress.org/Function_Reference/get_pages

get page template: https://codex.wordpress.org/Function_Reference/get_page_template