Why anything done on comments_array hook gets reset?

This is not a bug.

If you look at the comment starting at line 2100 of wp/wp-includes/comment-template.php in the wp_list_comments function, you’ll see that if the per_page argument is set and its value is different to that of the main wp_query, it’ll perform a separate comment query and display those, instead of the ones you have sorted.

To get around this you’ll need to create a custom walker and pass that to the wp_list_comments function and perform the sorting there:

wp_list_comments(['walker' => new MySortingCommentWalker()]);

Given that you are writing a plugin, you could use the wp_list_comments_args filter to ensure your custom walker is always used:

add_filter('wp_list_comments_args', function($args) {
    $args['walker'] = new MySortingCommentWalker();
    return $args;
});

You could extend the default Walker_Comment class, override the paged_walk method and sort the comments in there. Once sorted you can pass them to the parent method:

class MySortingCommentWalker extends \Walker_Comment
{
    public function paged_walk($elements, $max_depth, $page_num, $per_page, ...$args)
    {
        // Pass $elements to your sorting function here
        
        return parent::paged_walk($elements, $max_depth, $page_num, $per_page, $args);
    }
}