How to dynamically insert in the image caption?

That markup is created by the caption shortcode. If you look at the the source for that shortcode you will see this:

 // Allow plugins/themes to override the default caption template.
 $output = apply_filters('img_caption_shortcode', '', $attr, $content);
 if ( $output != '' )
  return $output;

That means that you can hook a function into img_caption_shortcode and completely hijack the caption shortcode. So copy the code from line ~643 to the end of the function, insert that into your own function, hook that to img_caption_shortcode, and make whatever changes you need to get the markup you want.