permalink error when modifying sanitize_title_with_dashes function

Why did you want to alter the URI here? SEO optimization?

An URI has a limited number of allowed chars, defined in an RFC. That’s the reason WordPress uses utf8_uri_encode to generate a permalink based on the posttitle.

You can find more information in this stackoverflow question:
Which characters make a URL invalid?

EDIT

I’ve checked the wp_insert_post() function in wp-includes/post.php, which generates a new post in the database based on the submitted $_POST array. There you have a call to sanitize_post() on line 2487 and also a call to sanitize_title() on line 2548. You should check both functions.

Another option could be, that an attached filter to one of these sanitize functions are fired in the background. You could remove any filters with the function remove_all_filters()

For example:

remove_all_filters('sanitize_title', 99);