You’re probably talking about theming wp_die(), which is the function that produces those grey error pages with a white box of text in them.
For a plugin solution, you could try this plugin, which says it does what you want. Not sure about version support though–it says it only works up to 3.1.4.
For a programatic solution, you’ll want to hook into the filter “wp_die_handler”. So you can do:
add_filter('wp_die_handler', 'my_die_handler');
As far as code for the my_die_handler function, you could start by looking at the default die handler — the function is called _default_wp_die_handler, and it starts on line 2796 of the core file /wp-includes/functions.php. You can copy the whole function to your plugin file (or your theme’s functions file), rename it my_die_handler, and customize it from there.