Ignore a filter on the media library

You can use is_admin() in conjunction with the admin global variable $pagenow to make sure you’re not on either the upload or media page:

function remove_images( $where ) {
    global $wpdb,
        $pagenow;

    if( is_admin() && in_array( $pagenow, array( 'upload.php', 'media-upload.php' ) ) {
        return $where
    }

    $where  .=  " AND {$wpdb->posts}.post_mime_type NOT LIKE 'image/%'";
    return $where;
}