How do I add /blog/ on my permalink without affecting the portfolio project types permalink?

You can use the “post_link” filter with a conditional to check the post type to modify the single posts.

For example:

add_filter( 'post_link', function($post_link, $id){
  $post = get_post( $id );
  if( is_object( $post ) && 'post' === $post->post_type ) {
    return home_url( '/blog/' . $post->post_name );
  }
  return $post_link;
}, 10, 2 );

You would also need to modify the rewrite rules using the ‘generate_rewrite_rules’ filter.

Edit: This question has already been answered with a better code example here: Custom permalink structure with a prefix just for posts