list posts and wrap them alphabetically A-Z

I would do it the other way around- loop over the alphabet and check all posts for a match with each letter.

$all_posts = new WP_Query(
    array(
        'orderby' => 'title',
        'order' => 'ASC',
        'posts_per_page' => -1
    )
);

if( $all_posts->have_posts() ){
    foreach( range( 'A', 'Z' ) as $letter ) {
        echo '<div class="group_letter"><div class="letter">' . $letter . '</div>';
        while( $all_posts->have_posts() ){
            $all_posts->the_post();
            $title = get_the_title(); 
            $initial = strtoupper( substr( $title, 0, 1 ) );
            if( $initial == $letter ){
                echo '<div class="post">' . $title . '</div>';
            }
        }
        $all_posts->rewind_posts();
        echo '</div>';
    }
}