Order get_terms using a Custom Field

Rather than outputting your terms in that initial loop, I would use it instead to build a new array, with your issue_date as the key:

$my_new_array = array( );
foreach ( $terms as $term ) {
    $issue_date = get_field( 'issue_date', $term );
    $my_new_array[$issue_date] = $term->name;
}

You can then loop through this new array in order:

ksort( $my_new_array, SORT_NUMERIC );

foreach ( $my_new_array as $issue_date => $term_name ) {
   echo "<li>" . $term_name . " " . $issue_date . "</li>";
}

This is untested.