How do I split a large query with a semi-expensive function included into multiple smaller queries

Yes, using 'posts_per_page' => -1 is a bad practice as it loads every matching post from the database, which can mean hundreds, thousands or millions of entries in the worst case.

And it looks like you only need post ID’s in your code, not the whole post objects. This means you can slim down the query by adding 'fields' => 'ids' to the arguments to only retrieve the required post ID’s.

In this case setting up the loop with have_posts() is also a bit unnecessary as the found posts can be accessed directly from the query object, e.g. $query->posts.

Instead of writing multiple queries and if statements, you can utilize a do-while loop with a paged query, which runs one or more times depending on how many pages of posts there are.

function wpse_410390_recount_versions_count() : void {
  $hasPosts = true;
  $paged = 0;
  
  do {
    $query = new WP_Query([
        'post_type' => 'release',
        'post_status' => ['publish', 'draft', 'future', 'private', 'pending'],
        'posts_per_page' => 100,
        'paged' => $paged,
        'tax_query' => array(
          array(
            'taxonomy' => 'release-format',
            'field' => 'slug',
            'terms' => 'album',
          ),
      ),
      'fields' => 'ids'
    ]);
  
    if ( $query->posts ) {
        foreach ( $query->posts as $post_id ) {
            some_function_to_update_versions_count($post_id);
        }
        $paged++;
    } else {
        $hasPosts = false;
    }
  } while ( $hasPosts );
}

Depending on your setup you might be able to toss in to the query arguments 'update_post_meta_cache' => false and 'update_post_term_cache' => false to slim down the query a bit more.

Related Posts: