Retrieve taxonomy terms in order of their post’s date?

I would use the terms_clauses filter, and then just use get_terms():

function wpse147412_order_terms_by_post_date( $pieces, $taxonomies, $args ) {
    global $wpdb;

    if ( 'post_date' !== $args['orderby'] ) {
        return $pieces;
    }

    $args = wp_parse_args( $args, array( 'post_types' => 'post' ) );

    $pieces['fields']   = 'DISTINCT ' . $pieces['fields'];
    $pieces['join']    .= " JOIN $wpdb->term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id";
    $pieces['join']    .= " JOIN $wpdb->posts AS p ON p.ID = tr.object_id";
    $pieces['where']   .= " AND p.post_type IN ('" . implode( "', '", (array) $args['post_types'] ) . "')";
    $pieces['orderby']  = 'ORDER BY p.post_date';

    return $pieces;
}
add_filter( 'terms_clauses', 'wpse147412_order_terms_by_post_date', 10, 3 );

....

$terms = get_terms( 'taxonomy', array( 'post_types' => 'series', 'orderby' => 'post_date', 'order' => 'DESC' ) );