Return Attachments from Custom Post Type

Your solution (edited incorrectly into the question) should be workable but you should be able to accomplish the same with fewer queries.

  1. Pull your portfolio IDs– note the fields argument.
  2. Then pull your attachments with those IDs as the parent post.
  3. Then you only loop over the one image array.

That works out to two primary queries. To wit:

$query = new WP_Query( 
  array( 
    'post_type' => 'portfolio', 
    'posts_per_page' => -1,
    'fields' => 'ids'
  ) 
);
$image_query = new WP_Query( 
  array( 
    'post_type' => 'attachment', 
    'post_status' => 'inherit', 
    'post_mime_type' => 'image', 
    'posts_per_page' => -1, 
    'post_parent__in' => $query->posts, 
    'order' => 'DESC' 
  ) 
);

if( $image_query->have_posts() ){
  while( $image_query->have_posts() ) {
      $image_query->the_post();
      $imgurl = wp_get_attachment_url( get_the_ID() );
      echo '<div class="portfolio-item">';
      echo '<a href="'.$imgurl.'" rel="lightbox[portfolio-home]"><img class="portfolio" src="'.$imgurl.'"></a>';
      echo '</div>';
  }
}

The way you are doing it would mean one query for the portfolio posts, plus another image query for each portfolio result. That could mean dozens, hundreds, or even more queries depending on the size of your database.