Can I use images as anchor tags with

Well, yes, of course. Take a look at this reference:

http://codex.wordpress.org/Function_Reference/previous_post_link
http://codex.wordpress.org/Function_Reference/next_post_link

You can customize it however you like, so you can simply put an html image tag in there. The function must be used in the loop to work. I’d do it like this:

<?php previous_post_link('%link', '<img src="https://wordpress.stackexchange.com/questions/141544/Image URL" width="20" height="20" alt="%title" />'); ?>

The first parameter controls what comes before and after the link. Because we don’t want anything before or after the link, but within, it’s just the link.

The second parameter is crucial. It says what text should be linked. Here, instead of text we put in the image we want. I’ve also added the post title as alt attribute.

Same way applies to the next link 🙂