Ok, thank you all who wanted to help.
My final code (based on https://goo.gl/c55S3v and it is working perfectly):
$this_tag = get_queried_object();
$all_tags = get_tags();
foreach( $all_tags as $position => $tag ) :
if( $this_tag->term_id == $tag->term_id ) :
$next_tag_pos = $position + 1;
$prev_tag_pos = $position - 1;
break;
endif;
endforeach;
$prev_tag_pos = $prev_tag_pos < 0 ? count($all_tags) - 1 : $prev_tag_pos;
$next_tag_pos = $next_tag_pos > count($all_tags) - 1 ? 0 : $next_tag_pos;
echo("<script>console.log('Previous tag: ". $prev_tag_pos . " ". get_term_link( $all_tags[$prev_tag_pos] ) . "');</script>");
echo("<script>console.log('Next tag: ". $next_tag_pos . " ". get_term_link( $all_tags[$next_tag_pos] ) . "');</script>");