Use the following instead of add_post_meta:
update_post_meta($post_id, 'when' $date);
add_post_meta
does just that, adds a new meta key every time, including duplicates in your case.
update_post_meta
also incorporates add_post_meta
if a key doesn’t exist, but updates an existing key.
For reference: http://codex.wordpress.org/Function_Reference/update_post_meta
If you want to store rolling dates in an array, you would simply create the array before storage like so:
$dates = get_post_meta($post_id, 'when', true);
if(is_array($dates))
$dates[] = $new_date; //I'm sure you would do more processing here
else
$dates = array($new_date);
update_post_meta($post_id, 'when', $dates);
I would recommend that you create a custom interface for this rather than using the default Custom Fields interface and rename your custom field to ‘_when’ so it doesn’t appear in the main interface. Since you’re saving an array, it wouldn’t be really practical to display it to the user in raw form.
Check out these two references for more info in creating a meta box: