How best to display notifications on the front-end using caught exceptions as an example

For admin area you can use admin_notices:

https://codex.wordpress.org/Plugin_API/Action_Reference/admin_notices

For public area (front end) you would probably need to output to some hook and insert the proper CSS. You could use wp_footer() for that, which is present in every theme (or at least should be) and is inside the body tag.