Modify Term Update Redirection

I agree with you, it’s somewhat an unexpected user experience, when you’re redirected after updating a term.

Additionally there seems to be no error handling on the form itself, if you try to make some errors, like deleting the term name and slug, it will not show any errors, just ignore your changes and redirect.

You can try the following to stay on the same edit term page after updating:

/**
 * Stay on the edit term page after updating.
 * @see http://wordpress.stackexchange.com/a/168206/26350
 */

add_filter( 'wp_redirect', 
    function( $location ){
        $mytaxonomy = 'post_tag'; # <-- Edit this to your needs!
        $args = array(
            'action'   => FILTER_SANITIZE_STRING,
            'taxonomy' => FILTER_SANITIZE_STRING,
            'tag_ID'   => FILTER_SANITIZE_NUMBER_INT,
        );
        $_inputs    = filter_input_array( INPUT_POST, $args );
        $_post_type = filter_input( INPUT_GET, 'post_type', FILTER_SANITIZE_STRING );
        if( 'editedtag' === $_inputs['action'] 
            && $mytaxonomy === $_inputs['taxonomy']
            && $_inputs['tag_ID'] > 0
        ){
            $location = add_query_arg( 'action',   'edit',               $location );
            $location = add_query_arg( 'taxonomy', $_inputs['taxonomy'], $location );
            $location = add_query_arg( 'tag_ID',   $_inputs['tag_ID'],   $location );
            if( $_post_type )
                $location = add_query_arg( 'post_type', $_post_type, $location );
        }
        return $location;
    }
);

where you have to modify the $mytaxonomy value to your needs.

You might also want to add a Go back link to the form, for example:

/**
 * Add a "Go back" link to the post tag edit form.
 * @see http://wordpress.stackexchange.com/a/168206/26350
 */

add_action( 'post_tag_edit_form',
    function( $tag ) {
        $url = admin_url( 'edit-tags.php' );
        $url = add_query_arg( 'tag_ID',   $tag->term_id, $url );
        $url = add_query_arg( 'taxonomy', $tag->taxonomy, $url );
        printf( '<a href="https://wordpress.stackexchange.com/questions/168193/%s">%s</a>', $url, __( 'Go back' ) );
    }
);

where we use the {$taxonomy}_edit_form hook. It would probably make more sense to add the link directly under the submit button, or in the same line, not above it like we do here, but I couldn’t find any suitable hook for that. You could of course use CSS to change the link’s position.