What’s the purpose / logic of split_the_query in WP_Query->get_posts

It is all for adding posts into the cache. The usefulness of this is so that get_post( $post_id ) doesn’t result in another hit to the database. This is likely as many of the template functions inside the loop call get_post().

I don’t fully understand the logic for expression for $split_the_query. The limit makes sense as a large limit is probably some batch processes instead being displayed by the theme.

The following code is conditional logic:

if ( $split_the_query ) {
        // First get the IDs and then fill in the objects

        $this->request = "SELECT $found_rows $distinct $wpdb->posts.ID FROM $wpdb->posts $join WHERE 1=1 $where $groupby $orderby $limits";

        $this->request = apply_filters( 'posts_request_ids', $this->request, $this );

        $ids = $wpdb->get_col( $this->request );

        if ( $ids ) {
            $this->posts = $ids;
            $this->set_found_posts( $q, $limits );
            _prime_post_caches( $ids, $q['update_post_term_cache'], $q['update_post_meta_cache'] );
        } else {
            $this->posts = array();
        }
    } else {
        $this->posts = $wpdb->get_results( $this->request );
        $this->set_found_posts( $q, $limits );
    }

$ids = $wpdb->get_col( $this->request ); This gets us an array of the IDs that were returned by the call.

if ( $ids ) If this is false, then no posts returned so just set the posts to an empty array.

_prime_post_caches( $ids, $q['update_post_term_cache'], $q['update_post_meta_cache'] ); This tells WordPress cache to capture the new post IDs for the cache. You’ll notice you have two query parameters you can pass in the query associated array to tell which caches to update.

It is weird that a custom post type would cause an issue while the default post post type would give you no issue. You may just want to set the two query vars to false to prevent caching.

It does look like other people are having your issue when I do a Google search for update_post_caches out of memory.