How to use custom footer template in a site-plugin?

You cannot override header / footer templates from a plugin, you’ll have to change the theme.

theme_page_templates can be used to change the general template for the post type page. That template then uses get_header() and get_footer(), which in turn include header.php / footer.php (optionally header-$name.php, if a parameter has been passed to get_header(), same for get_footer()).