What you need to do is change the excluded_terms argument of the previous_post_link and next_post_link functions which you called $catagory. Simply remove all the category ids of your current post from that excluded_terms array. Make sure to define an array of categories which you don’t want to be displayed like you did it in your example above.
$catagory = array(3,5,6,4,1) // array of category ids you don't want to be displayed
the next step is to find all the category ids of your current post and put it in a new array.
$post_category_objects = get_the_category(); // returns an array of WP_Term objects
$post_category_ids = array();
foreach($post_category_objects as $pco){
array_push($post_category_ids, $pco->term_id); // adds the post's category id to the $post_category_ids array
}
next you need to remove the category ids of the current post from $catagory (<– the array which contains the values to be excluded from the previous_post_link and next_post_link functions).
Assuming the post has the categories 6 and 7, these category ids will be removed from $catagory.
// removes the current post's category ids from the $catagory-array
foreach( $post_category_ids as $pci ){
$key = array_search( $pci, $catagory );
if( $key !== false ){
unset( $catagory[$key]);
}
}
// rearranges the $catagory's keys
$catagory = array_values($catagory);
Put the whole code above in your if-statement and it should work.