I would hook into the wp_head
action. I would place this in a plugin so as to abstract it from your presentation layer. This allows for scalability and changing of themes. This also prevents any analytics collateral damage if a step is missed in migration from one theme to the next.
add_action('wp_head', 'wpse_43672_wp_head');
function wpse_43672_wp_head(){
//Close PHP tags
?>
ADD YOUR PLAIN HTML CODE HERE
<?php //Open PHP tags
}