Output the slug and name of a CPT single post taxonomy term

If you’re on a single post, then your code won’t work, because get_queried_object will contain the post (and not a term).

If you want to get terms of given post, you should use get_the_terms function.

$terms = get_the_terms( get_the_ID(), '<TAXONOMY_NAME>' ); // <- put real tax name in here

You also have to remember, that a post may have multiple terms assigned to it, so the line above will get you an array of terms. You can loop through it like so:

foreach ( $terms as $term ) {
     echo '<span class="back-button"><a href="https://wordpress.stackexchange.com/portfolio/" .$term->slug. '">&larr; Back to ' .$term->name. '</a></span>';
}

And the last thing…

You should never build term links like you do. There is a function for that to: get_term_link and you should use it instead of concatenating slug of a term with some strings (why? Because some plugins may change the structure of such links and your code will stop working then).

So the full code can look like this:

$terms = get_the_terms( get_the_ID(), '<TAXONOMY_NAME>' ); // <- put real tax name in here
if ( $terms and ! is_wp_error( $terms ) ) {
    foreach ( $terms as $term ) {
        echo '<span class="back-button"><a href="' . esc_attr( get_term_link( $term ) . '">&larr; Back to ' . esc_html($term->name) . '</a></span>';
    }
}