How to limit number of images being printer out in “Set Featured Image” pop up?

Update 1:

After dived into core AJAX call, this filter will only happen on post.php page:

add_filter('ajax_query_attachments_args', function($query){
    if ( isset($_POST['post_id']) && !empty($_POST['post_id']) ) {
        $query['posts_per_page'] = 10; // output 10 images only.
    }
    return $query;
});

1. You can use ajax_query_attachments_args filter:

add_filter('ajax_query_attachments_args', function($query){
    $query['posts_per_page'] = 10; // output 10 images only.
    return $query;
});

2. Because of querying attachments happens in core and using AJAX, I don’t think we can optimize the whole thing.