How to tax query X number of posts related by tag first, then by category if not enough in tag-related

You cannot do this in one query, this is a bit too advanced for what WP_Query can do at this point in time. We will need to run at least two (I’m going to use 3) queries here to achieve what you want

FIRST QUERY

The first query will query for posts according to post_tag, that will also exclude the current post. We will only get post ID’s

SECOND QUERY

This query will handle the “fill-up” posts which will come from the category taxonomy. For this query to be successful, we will need the following

• Exclude the current post and the posts from the first query to avoid duplicate posts

• Get the amount of posts in the first query, subtract that from 4, and then use that diffrence to set the amount of posts that should be retrieved by the this query

We will also just get post ID’s here

THIRD QUERY

In order to keep the integrity from the query object, we will combine all the ID’s from the previousquery and query the final post objects. You might think this is expensive, but it is not. Because the first two queries uses get_posts() and only get post ID’s, they are really super fast. By querying only ID’s, we also do not updates caches which makes these queries even faster

SOLUTION

I prefer creating a function for such big pieces as code as it keeps my templates clean. Please note, the following is untested and requires PHP 5.4

function get_max_related_posts( $taxonomy_1 = 'post_tag', $taxonomy_2 = 'category', $total_posts = 4 )
{
    // First, make sure we are on a single page, if not, bail
    if ( !is_single() )
        return false;

    // Sanitize and vaidate our incoming data
    if ( 'post_tag' !== $taxonomy_1 ) {
        $taxonomy_1 = filter_var( $taxonomy_1, FILTER_SANITIZE_STRING );
        if ( !taxonomy_exists( $taxonomy_1 ) )
            return false;
    }

    if ( 'category' !== $taxonomy_2 ) {
        $taxonomy_2 = filter_var( $taxonomy_2, FILTER_SANITIZE_STRING );
        if ( !taxonomy_exists( $taxonomy_2 ) )
            return false;
    }

    if ( 4 !== $total_posts ) {
        $total_posts = filter_var( $total_posts, FILTER_VALIDATE_INT );
            if ( !$total_posts )
                return false;
    }

    // Everything checks out and is sanitized, lets get the current post
    $current_post = sanitize_post( $GLOBALS['wp_the_query']->get_queried_object() );

    // Lets get the first taxonomy's terms belonging to the post
    $terms_1 = get_the_terms( $current_post, $taxonomy_1 );

    // Set a varaible to hold the post count from first query
    $count = 0;
    // Set a variable to hold the results from query 1
    $q_1   = [];

    // Make sure we have terms
    if ( $terms_1 ) {
        // Lets get the term ID's
        $term_1_ids = wp_list_pluck( $terms_1, 'term_id' );

        // Lets build the query to get related posts
        $args_1 = [
            'post_type'      => $current_post->post_type,
            'post__not_in'   => [$current_post->ID],
            'posts_per_page' => $total_posts,
            'fields'         => 'ids',
            'tax_query'      => [
                [
                    'taxonomy'         => $taxonomy_1,
                    'terms'            => $term_1_ids,
                    'include_children' => false
                ]
            ],
        ];
        $q_1 = get_posts( $args_1 );
        // Count the total amount of posts
        $q_1_count = count( $q_1 );

        // Update our counter
        $count = $q_1_count;
    }

    // We will now run the second query if $count is less than $total_posts
    if ( $count < $total_posts ) {
        $terms_2 = get_the_terms( $current_post, $taxonomy_2 );
        // Make sure we have terms
        if ( $terms_2 ) {
            // Lets get the term ID's
            $term_2_ids = wp_list_pluck( $terms_2, 'term_id' );

            // Calculate the amount of post to get
            $diff = $total_posts - $count;

            // Create an array of post ID's to exclude
            if ( $q_1 ) {
                $exclude = array_merge( [$current_post->ID], $q_1 );
            } else {
                $exclude = [$current_post->ID];
            }

            $args_2 = [
                'post_type'      => $current_post->post_type,
                'post__not_in'   => $exclude,
                'posts_per_page' => $diff,
                'fields'         => 'ids',
                'tax_query'      => [
                    [
                        'taxonomy'         => $taxonomy_2,
                        'terms'            => $term_2_ids,
                        'include_children' => false
                    ]
                ],
            ];
            $q_2 = get_posts( $args_2 );

            if ( $q_2 ) {
                // Merge the two results into one array of ID's
                $q_1 = array_merge( $q_1, $q_2 );
            }
        }
    }

    // Make sure we have an array of ID's
    if ( !$q_1 )
        return false;

    // Run our last query, and output the results
    $final_args = [
        'ignore_sticky_posts' => 1,
        'post_type'           => $current_post->post_type,
        'posts_per_page'      => count( $q_1 ),
        'post__in'            => $q_1,
        'order'               => 'ASC',
        'orderby'             => 'post__in',
        'suppress_filters'    => true,
        'no_found_rows'       => true
    ];
    $final_query = new WP_Query( $final_args );

    return $final_query;
}

You can now use the function as follow in your single template

$query = get_max_related_posts();
if ( $query ) {

    while ( $query->have_posts() ) {
        $query->the_post();

            echo get_the_title() . '</br>';

    }
    wp_reset_postdata();
}

FEW NOTES

• The defaults are already set to post_tag, category and 4 for the three parameters respectively, so you do need need to pass any values to the function when calling it

• If you need to swop the taxonomies around or set different taxonomies or set the posts per page to anything else than 4, simply pass them in the correct order to the function

  $query = get_max_related_posts( 'tax_1', 'tax_2', 6 );