Can I add a widget to the of my site?

I took a closer look at the answer Mayeenul suggested, and with a combination of that and the top answer I got it to work. Seems like a custom plugin is the way to go, and much easier than I expected to implement.

First I created a folder at inside wp-content/plugins/sv-custom-head (using my initials to avoid conflicts) then added one plugins.php file, with this content:

<?php
/*
Plugin Name: SV Custom Head
Description: Description
Version: 1.0
Author:
Author URI:
Plugin URI:
*/

function sv_custom_head()
{
?>
    <script>
    (my code)
    </script>
<?php
}

add_action('wp_head', 'sv_custom_head');