Unfortunatelly, I didn’t understand your question right, but I see only 2 possible problems here. Here you will need to use:
$blog->current_post – current post index
$blog->post_count – total posts
If you need to wrap each 4 posts with a div, and if there are less then 4 posts in the end, wrap them to. To achieve this you need to add another OR condition to the second if statement. Check if the current post is equal to the last post.
//before...
if( $blog->current_post % 4 == 3 )
//after...
if( $blog->current_post % 4 == 3 || $blog->current_post == $blog->post_count - 1)
If you want to wrap each 4 posts in a div and “leave” the remainder, you can add another if statement, which should go first, just after while loop starts.
//your while loop starts
while ( $blog->have_posts() ) : $blog->the_post();
//break the loop if there are less than 4 posts left
if( $blog->current_post >= $blog->post_count - ($blog->post_count % 4)){
break;
}