Pagination for a category

The problem exists because you are using query_posts(), instead of properly filtering the main query via pre_get_posts.

Remove your query_posts() call from the template file, then add the following to functions.php:

function wpse74325_pre_get_posts( $query ) {
    if ( $query->is_main_query() && is_category( 5 ) ) {
        $query->set( 'posts_per_page', '20' );
    }
}
add_action( 'pre_get_posts', 'wpse74325_pre_get_posts' );

This function will properly modify the main query object, before retrieving posts, allowing WordPress to assign pagination properly.