complex restriction of items in media library

Good code. I think you could simplify by making direct SQL queries instead:

-- Featured Images
SELECT `meta_value` FROM `wp_postmeta` WHERE `meta_key` = '_thumbnail_id';
--> $thumbnail_ids

-- Header Images
SELECT `post_id` FROM `wp_postmeta` WHERE `meta_key` = '_wp_attachment_context' and `meta_value` = 'custom-header';
--> $header_ids

Also, with your method, I think that if you call WP_Query with suppress_filters => true (or use get_posts() instead; it suppresses filters by default) you can avoid having to remove your own filter.