How best to apply do_shortcode in media.php for captions

Caption shortcode attributes are merged with defaults using shortcode_atts function like so (see source in media.php):

$atts = shortcode_atts( array(
  'id'  => '',
  'align'   => 'alignnone',
  'width'   => '',
  'caption' => '',
  'class' => '',
), $attr, 'caption' );

So the 3rd $shortcode param is in use with the value of 'caption'.

As you can see in shortcode_atts source code, it means that the filter

"shortcode_atts_caption"

will be fired, allowing you to modify the attributes that will be used.

In your case you have to do something like:

add_filter("shortcode_atts_caption", function($atts) {
  if (isset($atts['caption'])) {
    $atts['caption'] = do_shortcode($atts['caption']);
  }
  return $atts;
});

Please note: if the caption will contain the shortcode 'caption' this will cause an endless loop.

This can be avoided removing filter before to call do_shortcode on caption:

function my_caption_shortcode($atts) {
  if (isset($atts['caption'])) {
    // avoid endless loop
    remove_filter( current_filter(), __FUNCTION__);
    // apply shortcodes
    $atts['caption'] = do_shortcode($atts['caption']);
    // restore filter
    add_filter(current_filter(), __FUNCTION__);
  }
  return $atts;
}

add_filter("shortcode_atts_caption", "my_caption_shortcode");