Why is WP template_include overwritting all templates rather than specified page?

Thanks to @WebElaine’s comment. I needed to move the template return outside of the function so that it looks like this:

public function add_my_template($template){
  if ( is_page( 'my-unique-page' ) ) {
    $template = plugin_dir_path( __FILE__ ) . 'page-my-unique-page.php' ;
  }
  return $template;
}