When I remove get_header() from 404 page, my css doesnt work

Of course it doesn’t. When you remove get_header() you also remove the code that loads the CSS files, not to mention that you remove large blocks of necessary HTML markup. The template you’ve posted above is very, very broken.

You’ve also left out get_footer() which will most likely leave your markup broken as well.

Instead of leaving out get_header() use it with an optional argument to load a truncated header file for your 404 page:

Description

Includes the header.php template file from your current theme’s
directory. If a name is specified then a specialised header
header-{name}.php will be included.

If the theme contains no header.php file then the header from the
default theme wp-includes/theme-compat/header.php will be included.

For example (from the Codex):

if ( is_home() ) :
    get_header( 'home' );
elseif ( is_404() ) :
    get_header( '404' );
else :
    get_header();
endif;

In your template, you only need get_header( '404' ); plus a PHP file named header-404.php containing whatever you want to display.