How to inject html to every page with a plugin?

You can use the is_page() function or is_single() function to echo content to front end pages

function add_content_to_page {

if ( is_page() ) {
echo '<p>content</p>';
 }   
}

or combine the two :

function add_content_to_page {

if ( is_page() || is_single() ) {
echo '<p>content</p>';
 }   
}

Similarly you can do is_page(41) to target a specific page id
or is_home() || is_front_page() to target the homepage alone
there a lot more options and examples in the WordPress codex