How to modify category.php to list posts alphabetically?

Try to do all theme modifications in functions.php whenever possible. It keeps the theme files clean and uncluttered. Here’s an example using the pre_get_posts action:

function order_category_archives( $query ) {
  if ( is_category() && $query->is_main_query() ){ // is_category() can specify a category, if necessary
    $query->set( 'orderby', 'title' );
    $query->set( 'order', 'ASC' );
  }
}

add_action( 'pre_get_posts', 'order_category_archives' );

Notice that this uses is_category() to modify the query only if we’re looking at a category archive page. An optional parameter can be added to specify which category (or categories) this should be limited to.

Finally, we use is_main_query() to avoid modifying any additional queries that may be on the page.