Problem solved. It was because I had the Custom Post Type and the Page name the same, “Portfolio” for both. Once I changed this, I went to Settings > Permalinks and flushed. Re-Saved Links. Then checked all the posts links that was using the CPT, and everything went fine after that. I did use the edit from Breezer as well but it worked with or without the edit.
