I ended up adding an additional taxonomy meta class
I was then able to use the following to get my phone number:
<?php
//Get the correct taxonomy ID by slug
$term = get_term_by( 'slug', get_query_var( 'term' ), get_query_var( 'taxonomy' ) );
//Get Taxonomy Meta
$saved_data = get_tax_meta($term->term_id,'loc_phone');
?>
<?php if( $saved_data != "" ) {
echo $saved_data;
} else {
echo '(555) 555-555';
}?>