How can I reduce amount of ifs and else ifs in this specific block of code?

You are doing it all wrong

  • get_term() is the wrong function to use to get terms attached to a post. get_term() is used to get the term object from the db for a given term ID. The first parameter accepts the term ID, and not the post ID. You want to be using get_the_terms() to get the terms attached to a post

  • You really do not need any if/else statements here, simply pass the term ID/slug/name as image name depending on how your images are named. For the purpose of answering the eact examples as in OP, I have used the term id as image name. Change this to your exact needs.

  • As already stated in another answer, you need == to compare two values, = assigns a value to another or to a variable

As you have stated has_term() works, so we can do the following then to solve the issue: (I have written a function which you can add in functions.phpo and then call the function inside the loop where needed)

NOTE:

All the code is untested and might be buggy. Be sure to test it on a test install with debug set to true. The code also requires a minimum of PHP 5.4

function get_term_related_image( $taxonomy = 'category' )
{
    // Invoke the global $post object
    global $post;

    // Check if we have a taxonomy and if it is valid, else return false
    if ( !$taxonomy )
        return false;    

    if ( $taxonomy !== 'category' )
        $taxonomy = filter_var( $taxonomy, FILTER_SANITIZE_STRING );

    // Get the terms assigned to the post
    $terms = get_the_terms( $post->ID, $taxonomy );

    // Make sure we actually have terms, if not, return false
    if (    !$terms
         || is_wp_error( $terms )
    )
        return false;

    // Get the first term from the array of terms
    $term = $terms[0];

    // Get the image
    $image = get_stylesheet_directory_uri() . '/images/review-scores-thumbs/' . $term->term_id . '.jpeg';

    // Check if we actually have an image before returning it
    if ( !getimagesize( $image ) ) 
       return false;

    // We have made it, YEAH!!!, return the image
    $permalink  = get_the_permalink( $post );
    $title_attr = the_title_attribute( ['echo' => false, 'post' => $post] );

    return '<a href="' . $permalink . '" title="' . $title_attr . '"><img src="' . $image . '"/></a>';
}

Inside the loop, you can just call the function as follow: (Just remember to pass the taxonomy name as first parameter if you are not using build in taxonomy category)

echo get_term_related_image( 'review-score' );

EDIT:

The code is now tested and works as expected