List all categories but exclude current post category on single post page

The idea here would be to use the queried object’s ID to get the post terms (because we are most probably outside the loop here, if inside the loop, just use get_the_ID()). From there, we can use wp_list_pluck() to get all the term ID’s and simply pass that to get_terms() exclude parameter

Just a note, as from WordPress V4.5, the taxonomy should be passed as an arguments of $args, I’ll handle both cases

PRE 4.5

// Set all our variables
$taxonomy = 'category';
$post_id  = $GLOBALS['wp_the_query']->get_queried_object_id();
$args     = [
    'hide_empty' => false
];

// Get the ID's from the post terms
$post_terms = get_the_terms( $post_id, $taxonomy );
if (    $post_terms
     && !is_wp_error( $post_terms )
) {
    $term_ids = wp_list_pluck( $post_terms, 'term_id' );

    // Get all the terms with the post terms excluded
    $args['exclude'] = $term_ids;
}

$terms = get_terms( $taxonomy, $args );

V 4.5 + version

// Set all our variables
$taxonomy = 'category';
$post_id  = $GLOBALS['wp_the_query']->get_queried_object_id();
$args     = [
    'taxonomy'   => $taxonomy,
    'hide_empty' => false
];

// Get the ID's from the post terms
$post_terms = get_the_terms( $post_id, $taxonomy );
if (    $post_terms
     && !is_wp_error( $post_terms )
) {
    $term_ids = wp_list_pluck( $post_terms, 'term_id' );

    // Get all the terms with the post terms excluded
    $args['exclude'] = $term_ids;
}

$terms = get_terms( $args );