sanitize vimeo embed code?

You need to add a custom validation/sanitization callback, and hook it into publish_post (and/or draft_post and/or future_post, as applicable). For example:

<?php
function wpse_44807_update_custom_post_meta() {
    // Globalize the $post object
    global $post;
    // If our custom post meta key is set, sanitize it;
    // otherwise, return false
    $my_post_custom = ( isset( $_POST['_my_post_custom'] ? wp_filter_nohtml_kses( $_POST['_my_post_custom'] ? false );
    // Now, delete or update our custom post meta key
    if ( false == $my_post_custom ) {
        delete_post_meta( $post->ID, '_my_post_custom' );
    } else {
        update_post_meta( $post->ID, '_my_post_custom', $my_post_custom );
    }
}
add_action( 'publish_post', 'wpse_44807_update_custom_post_meta' );
add_action( 'draft_post', 'wpse_44807_update_custom_post_meta' );
?>

Note that I’m sanitizing using the wp_filter_nohtml_kses() filter, which would be appropriate if you are expecting, say, a video ID or something similarly alpha-numeric. Your choice of sanitization will change, depending on the type of expected input.

Also: I’m using an underscore-prefixed custom post meta key, which is appropriate if you’re defining a custom post meta box for your custom post meta key. (The underscore prefix hides this meta key from the generic “custom field” meta box drop-down.)