I want to change the default HTML code structure so that I can insert additional tags

Run a filter on img_caption_shortcode, you can find that hook in the source here.
http://core.trac.wordpress.org/browser/tags/3.0.1/wp-includes/media.php#L720

Example

add_filter( 'img_caption_shortcode', 'my_caption_html', 10, 3 );
function my_caption_html( $current_html, $attr, $content ) {
    extract(shortcode_atts(array(
        'id'    => '',
        'align' => 'alignnone',
        'width' => '',
        'caption' => ''
    ), $attr));
    if ( 1 > (int) $width || empty($caption) )
        return $content;

    if ( $id ) $id = 'id="' . esc_attr($id) . '" ';

    return '<div ' . $id . 'class="wp-caption ' . esc_attr($align) . '" style="width: ' . (10 + (int) $width) . 'px">'
. do_shortcode( $content ) . '<p class="wp-caption-text">' . $caption . '</p></div>';
}

I’ve basically borrowed most of the code from the original function to use in the filter, but it should give you something to build on top of…

Hope that helps.. 🙂