That code should work. I just tested in my own index.php and it worked. Are you sure you’re looking at the index.php file. Sometimes WP actually pulls a different template: For more info, see WP template hierarchy diagram. To test what template you’re seeing, place this code in your functions.php file temporarily:
add_action('wp_head', 'show_template');
function show_template() {
global $template;
echo '<span style="color: #000; z-index: 10000; position: absolute;">' . $template . '</span>';
}