This happens because the_title filter is run when the the_title() or related function is called to print the title, and prev/next links functions run the same the_title filter.
But, the problem here is that you are having general conditions that main post and prev/next fulfil: singular, expired post status and loop. To target the main post only in the single template add condition $post_id == get_the_ID() to your IF – $post_id is the filter function argument, and get_the_ID() get’s the ID of the global $post.