There are a lot of questions here regarding pagination, it is definitely one of the least understood aspects of how WordPress works internally.
To understand why you get 404s, we’ll start by looking at the Action Reference in Codex to see the process WordPress follows for each request.
The process begins by loading up the plugins and theme, and doing some initialization to set everything up, etc., etc.. The part we are interested in begins at parse_request, when WordPress determines what sort of page is being requested, and parse_query, when the query variables are set to generate the Main Query.
The next action is pre_get_posts, which gives us an opportunity to modify the main query. This is where you want to execute your own code to change the main query, for example, setting a different posts_per_page value (hint, hint).
Then we reach the wp action, after the main query is run, and finally the important bit of info in this explanation- the template_redirect action. This is the action where WordPress determines what template to load, based on the results of the main query. If it’s a category, the category template is loaded, if it’s an archive, the archive template is loaded, and if it’s a 404- the 404 template is loaded.
Yes, bold text means that last part was important, and hopefully we are beginning to understand the problem. WordPress decided the request is a 404 before your template was ever loaded. Whatever the results you get by running queries in the template are irrelevant. If the main query doesn’t return post(s), it’s a 404.
In your case, you’re getting a 404 because the main query is loading 5 posts per page, and you only have 3, there is no second page as far as the main query is concerned.
( But wait a minute, you say, why does it work on some pages and not others? Yes, confusingly enough it won’t always result in a 404, for example when the request is a singular page. )
The solution to your category problem was hinted at earlier in the story- if you want to modify the query, use pre_get_posts before the main query is run.
// modify main query for category ID 15 and all its children
function wpd_category_query( $query ) {
if( $query->is_main_query() && $query->is_category() ) {
$parent = 15; // your parent category
$categories = get_term_children( $parent, 'category' );
$categories[] = $parent;
if( is_category( $categories ) ) {
$query->set( 'posts_per_page', 2 );
}
}
}
add_action( 'pre_get_posts', 'wpd_category_query' );