Add custom fields when specific templates are selected

Can you do that? Absolutely! You simply need to query the _wp_page_template meta key value of the $post object, and act accordingly. Perhaps something like so:

// Globalize $post
global $post;
// Get the page template post meta
$page_template = get_post_meta( $post->ID, '_wp_page_template', true );
// If the current page uses our specific
// template, then output our post meta
if ( 'template-foobar.php' == $page_template ) {
    // Put your specific custom post meta stuff here
}

Now, I would recommend using a custom post meta box, rather than custom fields.

While full implementation of custom post meta boxes is slightly outside the scope of your question, the underlying answer remains the same. I’ll try to point you in the general direction, though. You’ll use a combination of add_meta_box(), called in a callback hooked into add_meta_boxes-{hook}, a callback to define the metabox, and a callback to validate/sanitize and save custom post meta.

function wpse70958_add_meta_boxes( $post ) {

    // Get the page template post meta
    $page_template = get_post_meta( $post->ID, '_wp_page_template', true );
    // If the current page uses our specific
    // template, then output our custom metabox
    if ( 'template-foobar.php' == $page_template ) {
        add_meta_box(
            'wpse70958-custom-metabox', // Metabox HTML ID attribute
            'Special Post Meta', // Metabox title
            'wpse70598_page_template_metabox', // callback name
            'page', // post type
            'side', // context (advanced, normal, or side)
            'default', // priority (high, core, default or low)
        );
    }
}
// Make sure to use "_" instead of "-"
add_action( 'add_meta_boxes_page', 'wpse70958_add_meta_boxes' );


function wpse70598_page_template_metabox() {
    // Define the meta box form fields here
}


function wpse70958_save_custom_post_meta() {
    // Sanitize/validate post meta here, before calling update_post_meta()
}
add_action( 'publish_page', 'wpse70958_save_custom_post_meta' );
add_action( 'draft_page', 'wpse70958_save_custom_post_meta' );
add_action( 'future_page', 'wpse70958_save_custom_post_meta' );

Edit

It may be better to wrap the entire add_meta_box() call in the conditional.