Change meta box when the page’s template is changed

function my_portfolio_meta_box_cb($post) {
     $template = get_post_meta($post->ID, '_wp_page_template', true);
     if ($template == 'portfolio.php') {
         add_meta_box(........);
     }
}

Hook this via the register_meta_box_cb arg when registering the post type or in your manually via add_meta_boxes-portfolio hook.