you are quering the same posts over and over, and that is way you are getting the same posts, to fix it just add
'paged' => get_query_var('paged') to your query arguments, so change:
<?php $portfolioloop = new WP_Query( array( 'post_type' => 'portfolio', 'posts_per_page' => 12 ) ); ?>
into:
<?php $portfolioloop = new WP_Query( array( 'paged' => get_query_var('paged'), 'post_type' => 'portfolio', 'posts_per_page' => 12 ) ); ?>
and just to make sure you are ok and to avoid errors add wp_reset_postdata();
at the end of your code.