Pagination hitting 404 page on /page/4

Just in case anyone ever stumbles across this, I found the solution.

First step was to create an archive.php page

In there I would access the loop.

The in functions.php

I have a category rewrite filter to update the category rewrite rules to use the following example url index.php?wpdmcategory=category-name&paged=2 when I hit /category-name/page/2

Previously, it was using the following example url, index.php?category_name=category-name&paged=2 so it was actually using the default post type for the category paging.

add_filter( 'category_rewrite_rules', 'vipx_filter_category_rewrite_rules' );`

function vipx_filter_category_rewrite_rules($rules) {
    $categories = get_terms('wpdmcategory', array("hide_empty" => false));

    if (is_array($categories) && !empty($categories)) {
        $slugs = array();
        foreach($categories as $category) {
            if (is_object($category) && !is_wp_error($category)) {
                if (0 == $category - > category_parent) {
                    $slugs[] = $category - > slug;
                } else {
                    $slugs[] = trim(get_category_parents($category - > term_id, false, "https://wordpress.stackexchange.com/", true), "https://wordpress.stackexchange.com/");
                }
            }
        }

        if (!empty($slugs)) {
            $rules = array();

            foreach($slugs as $slug) {
                $rules['('.$slug.
                    ')/feed/(feed|rdf|rss|rss2|atom)?/?$'] = 'index.php?wpdmcategory=$matches[1]&feed=$matches[2]';
                $rules['('.$slug.
                    ')/(feed|rdf|rss|rss2|atom)/?$'] = 'index.php?wpdmcategory=$matches[1]&feed=$matches[2]';
                $rules['('.$slug.
                    ')(/page/(\d+)/?)?$'] = 'index.php?wpdmcategory=$matches[1]&paged=$matches[3]';
            }
        }
    }
    return $rules;
}