display all posts from category with and without terms in chronological order

In the end I was thinking too complicated. Instead of having two tax_query arrays inside a single category why not just take the single category for starters and then filter out the posts that have terms with the use of conditionals. It now looks like this and works.

<?php 
$args = array(
    'post_type' => 'post',
    'tax_query' => array(
        array(
            'taxonomy' => 'category',
            'field'    => 'slug',
            'terms'    => array ('fruits'),                    
            ),
    ),
);
$catFruits = new WP_Query( $args );
    if($catFruits->have_posts()) :
        while($catFruits->have_posts()) : $catFruits->the_post();
        if (has_term('apples','nutrition')) { 
            // do stuff
        } elseif (has_term('oranges','nutrition')) { 
            //do stuff
        // If has not an array of the terms I don't want
        // Since I am in category fruits these are all posts without term
        ] elseif (!has_term('apples', 'oranges')){ 
            //do stuff 
        }
        endwhile;
    endif;
wp_reset_query();
?>

And in fact this is shorter and actually does the trick as well.

<?php 
$args = array(
    'post_type' => 'post',
    'category_name' => 'fruits',
);
$catFruits = new WP_Query( $args );
    if($catFruits->have_posts()) :
        while($catFruits->have_posts()) : $catFruits->the_post();
        if (has_term('apples','nutrition')) { 
            // do stuff
        } elseif (has_term('oranges','nutrition')) { 
            //do stuff
        // If has not an array of the terms I don't want
        // Since I am in category fruits these are all posts without term
        ] elseif (!has_term('apples', 'oranges')){ 
            //do stuff 
        }
        endwhile;
    endif;
wp_reset_query();
?>