Use the wp_sitemaps_taxonomies_query_args hook to set hide_empty to false:

add_filter(
    'wp_sitemaps_taxonomies_query_args',
    function ( $args ) {
        $args['hide_empty'] = false;
        return $args;
    }
);

Explanation

The list of terms in the site map is generated by the WP_Sitemaps_Taxonomies::get_url_list() method. The terms are queried in this method using the get_taxonomies_query_args() method to build the args for WP_Term_Query:

$args           = $this->get_taxonomies_query_args( $taxonomy );
$args['fields'] = 'all';
$args['offset'] = $offset;

$taxonomy_terms = new WP_Term_Query( $args );

If we look at this method, we see:

protected function get_taxonomies_query_args( $taxonomy ) {
    /**
     * Filters the taxonomy terms query arguments.
     *
     * Allows modification of the taxonomy query arguments before querying.
     *
     * @see WP_Term_Query for a full list of arguments
     *
     * @since 5.5.0
     *
     * @param array  $args     Array of WP_Term_Query arguments.
     * @param string $taxonomy Taxonomy name.
     */
    $args = apply_filters(
        'wp_sitemaps_taxonomies_query_args',
        array(
            'taxonomy'               => $taxonomy,
            'orderby'                => 'term_order',
            'number'                 => wp_sitemaps_get_max_urls( $this->object_type ),
            'hide_empty'             => true,
            'hierarchical'           => false,
            'update_term_meta_cache' => false,
        ),
        $taxonomy
    );

    return $args;
}

Of note, is that hide_empty is set to true. As per the documentation for WP_Term_Query::__construct():

hide_empty bool|int
Whether to hide terms not assigned to any posts. Accepts 1|true or 0|false. Default 1|true.

Thus, to include empty terms, we should add a function to the wp_sitemaps_taxonomies_query_args hook like:

add_filter(
    'wp_sitemaps_taxonomies_query_args',
    function ( $args ) {
        $args['hide_empty'] = false;
        return $args;
    }
);