Previous/next post links displaying wrong category than is desired

Your code problem is you get only the first category of a post because $cat[0] is the first, not current.

Inside The Loop:

<?php
previous_post_link( // prints the formatted link
    '<span>&laquo; %link</span>',  // beautifying around
    'Prev in this cat',            // the text (%link) to be used in the span above
    true,                          // confirm that you want only this category
    '',                            // no categories to exclude
    'category'                     // taxonomy name
);

echo " | "; // just a separator between next and prev

/* The same setup here, not shown to keep the answer clean */
next_post_link();

See previous_post_link() for parameters.

If you are outside of The Loop, let me know, I’ll update the answer and supply the additional code and comments.