For your first question- HTML support for captions was a new feature in wordpress 3.4, if you have any version prior to that, it won’t work
I’m not sure if i understood your 2nd question, but to exclude the caption from not displaying on the page, you’ll have to either delete the caption from the image or edit your theme’s template files
UPDATE
First if the image was inserted in the page content, it is inserted in the form of a shortcode, so you can use the filter img_caption_shortcode
to override the default shortcode output.
If it’s set as featured image, instead of using the_post_thumbnail
, use get_post_thumbnail_id
& generate the img
tag yourself & skip the caption.
Next to permanently disable the captions functionality(you already cleared you don’t want to do that), you can use this add_filter( 'disable_captions', '__return_true' );
If you want a WYSIWYG editor in captions, there isn’t any hook for that. You’ll have to code it yourself