You want the Widget Logic Plugin if you’re going to use widgets to do this. However, as you have guessed, this is not the best way to do this. I think the best way to achieve what you want to do is to create a custom widget that accesses the posts information if it detects that it’s on the correct page. Something like this:
global $wp_query;
$custom_data = get_post_meta( $wp_query->post->ID, 'your_custom_postmeta_key', true );
if( is_page() && !empty( $custom_data ) ){
echo $before_widget . $before_title . $title . $after_title;
echo apply_filters( 'the_content', $custom_data );
echo $after_widget;
}
If you stick that inside the normal widget structure, that would be pretty effective at creating a per-page widget that only appears if you’re on a page and that page has the meta content relevant to the widget. At that point, you could just use the custom fields that WordPress already provides on the pages to input the custom content.