Next/Previous Links in same category

previous_post_link takes 5 params, but you use only 2 of them.

Let’s take a look at other 3:

• in_same_term (boolean) (optional) Indicates whether previous post must
  be within the same taxonomy term as the current post. If set to
  ‘true’, only posts from the current taxonomy term will be displayed.
  If the post is in both the parent and subcategory, or more than one
  term, the previous post link will lead to the previous post in any of
  those terms. true false Default: false
• excluded_terms (string/array)
  (optional) Array or a comma-separated list of numeric terms IDs from
  which the next post should not be listed. For example array(1, 5) or
  ‘1,5’. This argument used to accept a list of IDs separated by ‘and’,
  this was deprecated in WordPress 3.3 Default: None
• taxonomy (string)
  (Optional) Taxonomy, if $in_same_term is true. Added in WordPress 3.8.
  Default: ‘category’

So if you want to navigate through posts from the same category, third param should be set to true.

<?php
    previous_post_link(
        '<div class="nav-next">%link</div>',
        '%title <span class="meta-nav">' . _x( '→', 'Previous post link', 'morphology' ) . '</span>',
         true
    );
?>

And if your portfolio uses custom taxonomy, you should point that out in last param:

<?php
    previous_post_link(
        '<div class="nav-next">%link</div>',
        '%title <span class="meta-nav">' . _x( '→', 'Previous post link', 'morphology' ) . '</span>',
         true,
         array(),
         'my_custom_taxonomy'
    );
?>

It works the same with next_post_link.