How to create A-Z index listing for custom post types?

Ok, you just have to make a loop with first letter check. WordPress query will handle the post_title ASC order.

EDIT : I use get_permalink() within the loop to get URL (https://developer.wordpress.org/reference/functions/get_permalink/)

$series = new WP_Query(array(
    'posts_per_page'        => -1,
    'post_type'             => 'series_type',
    'orderby'               => 'title',
    'order'                 => 'asc'
));
if($series->have_posts())
{
    $letter="";
    while($series->have_posts())
    {
        $series->the_post();

        // Check the current letter is the same that the first of the title
        if($letter != strtoupper(get_the_title()[0]))
        {
            echo ($letter != '') ? '</ul></div>' : '';
            $letter = strtoupper(get_the_title()[0]);
            echo '<div><ul><h4>'.strtoupper(get_the_title()[0]).'</h4>';            
        }

        echo '<li><a href="'.get_permalink().'">'.get_the_title().'</a></li>';
    }
}