Get posts from a top category and group by child categories

You are getting the “Category name” as much times as the numbers of post are in each category because you have the code to output the “Category name” inside the loop, so the “Category name” is printed in every loop itineration. You could fix this part, for example in this way:

<?php if ( have_posts() ) { ?>

    <h2><a>Name of "Subcategory"</a></h2>
    <p>Description of "Subcategory"</p>

    <?php
    //Now you can start the loop
    while (have_posts()) {
        the_post();
        //don't use include, use get_template_part() instead
        //include (TEMPLATEPATH . '/membresgim.php');
        get_template_part('membresgim');
    }
    ?>

<?php } else { ?>

    <p>Sorry, no posts matched your criteria.</p>

<?pph } ?>

You must know also that is recomended to avoid the use of query_posts() function for several reasons. In your case, as you are going to run several secondary loops, I think is better to use a new instance of WP_Query for each loop. A quick example (for what I understand from your question and comments):

<?php
//Change by the ID of your top categoy
$root_category = 45;
$subcategories =  get_categories('child_of=".$root_category);  
foreach  ($subcategories as $cat) {
    $args = array(
                category__in => array($cat->cat_ID),
            );
    $the_query = new WP_Query($args);
?>
    <section>
        <h2><a href="https://wordpress.stackexchange.com/questions/114718/<?php echo get_category_link($cat->cat_ID); ?>"><?php echo $cat->nicename; ?></a></h2>
    </section>
    <ul>
    <?php
    while($the_query->have_posts()) {
        $the_query->the_post();
        ?>
        <li><a href="<?php the_permalink(); ?>"><?php the_title(); ?></a></li>
    <?php } ?>
    <?php wp_reset_postdata(); ?>
    </ul>
<?php } ?>