Display on page every post in blog WITH comments

Short answer: Add comments_template(); under your the_content().

But there’s a better way to do this that will carry over any formatting and style you have on your main blog. Maybe you don’t want this, though, so ignore the rest if needed.

Here’s the improved loop code:

$args = array( 'numberposts' => -1);
$query = new WP_Query($args);

if ($query->have_posts()) {

    while ($query->have_posts()) {
        $query->the_post();
        get_template_part('content', get_post_format());
    }

    wp_reset_postdata();
} else {
    get_template_part('content', 'none');
}

That is a pretty basic WordPress loop, nothing really tricky going on and you almost had exactly that with your provided code. Instead of directly outputting the content, this code will load the template parts just like the loop in index.php, which should include the call to comments_template() if its a half-way decent theme.

Here’s the updated improved loop that should work for older themes that do not have content.php:

if ($query->have_posts()) {

    while ($query->have_posts()) {
        $query->the_post();
        the_content();
        comments_template();
    }

    wp_reset_postdata();
} else {
    echo 'No posts found.';
}

But your original code is literally doing the same thing, just organized a bit differently. I would suggest adding a wp_reset_postdata() call to your code, though, so you can restore the original query. Probably won’t hurt to leave it out, but it’ll be a mighty hard bug to track down in the future.