Maybe I’m misunderstanding the question – if so apologies!
»
represents the ยป
symbol.
I think all you need to do is add it into your second example, if that’s what’s displaying in your template:
<?php the_content("» Continue reading " . the_title('', '', false)); ?>
Hope this helps!