Get all posts that do not have a specific custom field without using a subquery

This is the way WordPress does it:

SELECT wp_posts.* FROM wp_posts
LEFT JOIN wp_postmeta ON ( wp_posts.ID = wp_postmeta.post_id AND wp_postmeta.meta_key = 'custom_field' )
WHERE wp_postmeta.post_id IS NULL

You can do this with WP_Query meta_query and 'compare' => 'NOT EXISTS'.