How to setup an automatic redirect for when there has been a critical error on the site?

The WP_Fatal_Error_Handler class allows for a drop-in file to override the default handling of errors. If you have a file at wp-content/php-error.php, that file will be used instead of the default WP error handling.

Note that this drop-in file comes with some caveats:

A drop-in php-error.php can be used as a custom template. This drop-in should control the HTTP status code and print the HTML markup indicating that a PHP error occurred. Note that this drop-in may potentially be executed very early in the WordPress bootstrap process, so any core functions used that are not part of wp-includes/load.php should be checked for before being called.

[emphasis added]

So you should be able to add wp-content/php-error.php with content something like this:

<?php
// wp_redirect() doesn't likely exist at this point,
// so we use PHP's `header()`.
header( 'Location: https://example.com/your-desired-url', true, 302 );
exit;

Note: The code above is untested.

References

• WP_Fatal_Error_Handler
• header()