Enqueue style for specific site type

is_tax() and is_404() perhaps?

https://codex.wordpress.org/Conditional_Tags

---

UPDATE:

I’m sorry my original answer was vague, to elaborate better:

You can add conditional tags in a wp_enqueue_scripts hook, choosing to register and enqueue when criteria is met, like so:

add_action( 'wp_enqueue_scripts', 'mythemes_scripts' );
function mythemes_scripts() {

    if (is_category()) {
        wp_register_style(
            'categoryStyles',  
            get_template_directory_uri() . '/style-category.css', 
            array(), // maybe the primary style.css handle here 
            filemtime( get_template_directory() . '/style-category.css' )
        );
        wp_enqueue_style( 'categoryStyles');
    }

    if (is_404()) {
        wp_register_style(
            'fourohfourStyles',  
            get_template_directory_uri() . '/style-404.css', 
            array(), // maybe the primary style.css handle here 
            filemtime( get_template_directory() . '/style-404.css' )
        );
        wp_enqueue_style( 'fourohfourStyles');
    }

}

The codex link above has many more conditional tags for situations like is_archive().

I’ve put "// maybe the primary style.css handle here" as doing so makes the main style sheet load before your conditional style sheets – so any CSS redefinitions are overwriting properly.

The filemtime() for version numbering (instead of ‘1.0’) will help with cache-breaking, which is valuable while trying to troubleshoot with multiple CSS includes that are presumably redefining.

And if this code is for a plugin instead of a theme you can swap get_template_directory_uri()/get_template_directory_uri() out for plugins_url()/plugin_dir_path() respectively.