Pagination not working

Question

As I already stated in comments, you should never use query_posts, only use query_posts if you are intended to break your page functionalities. Add query_posts to the very top of your EVIL LIST.

query_posts breaks the main query object, the very object that your pagination function relies on. Your pagination function relies on the $max_num_pages property from the main query. This is just one of the many things that query_posts break.

To solve your issue, make use of WP_Query as it seems that you are using a custom page template here. Just one note here, if home-page.php is a static front page, which I suspect from the template name, then

'paged' => ( get_query_var('paged') ? get_query_var('paged') : 1),

should be

'paged' => ( get_query_var('page') ? get_query_var('page') : 1),

as static front pages uses page and not paged

You can try something like this: (Very important, this is untested and copied and pasted from OP, and always remember to reset postdata once you are done with your custom query)

<?php
$args = array(
    'post_type' => 'custom_post_type',
    'posts_per_page' => 5,
    'paged' => ( get_query_var('paged') ? get_query_var('paged') : 1),
);
$query = new WP_Query($args);
?>
<table>
    <thead >
        <tr>

            <th>header</th>
            <th>header</th>
            <th>header</th>
            <th>header</th>

        </tr>
    </thead>
    <tbody>

        <?php  while ($query->have_posts()) : $query->the_post();?>
            <tr>

                <td><?php echo get_post_meta($post->ID, 'metakey', true); ?></td>
                <td><?php echo get_post_meta($post->ID, 'metakey', true); ?></td>
                <td><?php echo get_post_meta($post->ID, 'metakey', true); ?></td>    
                <td><?php echo get_post_meta($post->ID, 'metakey', true); ?></td>    

            </tr>
        <?php
        endwhile;
        wp_reset_postdata();
        ?>

    </tbody>
</table>

You then just need to update your pagination function with the $max_num_pages property rom your custom query

<?php
$big = 999999999; // need an unlikely integer

echo paginate_links( array(
    'base' => str_replace( $big, '%#%', esc_url( get_pagenum_link( $big ) ) ),
    'format' => '?paged=%#%',
    'prev_text'          => __(' Previous'),
    'next_text'          => __('Next '),
    'current' => max( 1, get_query_var('paged') ),
    'total' => $query->max_num_pages
) );
?>