Only allow the post author and admin to comment on a post

Depending on how your theme is set up, comments_open might be the filter you want. (in functions.php)

add_filter( 'comments_open', 'block_comments_for_others', 10, 2 );
function block_comments_for_others( $is_open, $post_id ) {

    $is_open = false;
    $post_author_id = get_post_field( 'post_author', $post_id );

    if ( get_current_user_id() == $post_author_id || current_user_can('administrator') )
        $is_open = true;

    return $is_open;
}

Or possibly something like this right in your theme where you display comments:

if ( get_current_user_id() == get_the_author_meta( 'ID' ) || current_user_can('administrator') ) {
    comments_template(); 
} else {
    echo "No commenting for you";
}