Trying to load different syles for 404.php page

You have 2 options here.

Use Built-in Classes

As you already mentioned, WordPress automatically adds classes to body, based on the current page. You can use it to style your elements differently, or even use your own different classes in your 404.php:

.error404 p {
    // Some different classes here
}

Enqueue Your Styles Only for 404 Page

You can check whether the page is a 404 error or not, and then enqueue your styles:

add_action('wp_enqueue_scripts','enqueue_my_script');
function enqueue_my_script(){
    if( is_page_template('404.php') ){
        // Enqueue your style here
    }
}