post_results filter confused by pagination in sidebar and main section

The answer is to specify all the args on the add_filter() call, and then you can have both the array of posts and the query sent to the callback.
add_filter( ‘posts_results’, ‘do_stuff’, 10, 2 );