Sort posts by category name and title

To get them broken down by Category, you need to loop through the list of categories and then query on each category:

$categories = get_categories( array ('orderby' => 'name', 'order' => 'asc' ) );

foreach ($categories as $category) {

   echo "Category is: $category->name <br/>";

   $catPosts = new WP_Query( array ( 'category_name' => $category->slug, 'orderby' => 'title' ) ); 

   if ( $catPosts->have_posts() ) {

       while ( $catPosts->have_posts() ) {
          $catPosts->the_post();
          echo "<a href="https://wordpress.stackexchange.com/questions/124037/the_permalink()">the_title()</a>";
       }

       echo "<p><a href="category/$category->slug">More in this category</a></p>";

   } //end if
  


} //end foreach

wp_reset_postdata();