why the same code got different results when using query_posts in functions.php and index.php

In your first code example, you hook into the pre_get_posts action, which fires during the main query. But instead of manipulating that query, you trigger another one using query_posts(), which fires the hook again, and you end up with an infinite loop.

With the second code example, you’re just overriding the main query after it’s occurred. But whilst it works, you’ve ended up with a redundant, time-consuming query.

You’re almost there, and props for your efforts, so let’s bring it together:

function wpse_103997_only_posts_by_current_user( $wp_query ) {
    if ( is_user_logged_in() && $wp_query->is_main_query() && $wp_query->is_home() )
        $wp_query->set( 'author', get_current_user_id() );
}
add_action( 'pre_get_posts', 'wpse_103997_only_posts_by_current_user' );