Writing LaTeX formulas with MathJax on WordPress site: the simplest way (possibly without affecting header.php)?

If you’re using any (non site-specific, custom made) theme without a child theme, then you shouldn’t modify the original theme files in any way. As any modifications will get wiped out the next time you update your theme.

If you’re using a child theme, you could copy the header.php file from your parent theme to your child theme directory. After that you could make any modifications to the copied file without fear of losing the customizations on the next (parent) theme update.

Another, and more appropriate, option when using a child theme would be to enqueue the custom script by adding necessary code to the functions.php file. More on that here, https://developer.wordpress.org/reference/hooks/wp_enqueue_scripts/

In this case however as the script is not exactly theme related then a better way than adding code to functions.php would be to enqueue the script in a custom plugin. https://developer.wordpress.org/plugins/intro/ This way you won’t lose the script or have to copy it over to another functions.php file, if you ever change your theme.

Basically you’d do the following,

1. Create new file with the name of mathjax-script-plugin.php (the name can be something else too, doesn’t really matter)
2. Add the code below to the file
3. Upload (via FTP or zip it and use the plugin installer) the file to wp-content/plugins
4. Activate the plugin from Dashboard > Plugins
5. Inspect your site’s source code to make sure the script is loaded properly

Plugin code,

<?php
/*
Plugin Name: MathJax
Version: 1.0.0
Description: Enqueue MathJax js
Author: Antti Koskinen
*/

function mathjax_enqueue_script() {   
  wp_enqueue_script( 
    'mathjax', // handle
    'http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML', // source
    array(), // depedencies
    false, // version
    true // in footer
  );  
}
add_action('wp_enqueue_scripts', 'mathjax_enqueue_script');