How can I show these pictures in two columns in my page?

If it’s a usable option for you, you can change the output of to use <span> tag instead of a <div>.

This how you’d change the output (to be added in theme functions.php) of the Caption shortcode:

// Source code from http://core.svn.wordpress.org/trunk/wp-includes/media.php

add_filter( 'img_caption_shortcode', 'wpse57262_cleaner_caption', 10, 3 );

function wpse57262_cleaner_caption( $output, $attr, $content ) {

    if ( ! isset( $attr['caption'] ) ) {
        if ( preg_match( '#((?:<a [^>]+>\s*)?<img [^>]+>(?:\s*</a>)?)(.*)#is', $content, $matches ) ) {
            $content = $matches[1];
            $attr['caption'] = trim( $matches[2] );
        }
    }

    extract(shortcode_atts(array(
        'id'    => '',
        'align' => 'alignnone',
        'width' => '',
        'caption' => ''
    ), $attr));

    if ( 1 > (int) $width || empty($caption) )
        return $content;

    if ( $id ) $id = 'id="' . esc_attr($id) . '" ';

    $output="<span " . $id . 'class="wp-caption ' . esc_attr($align) . '" style="width: ' . (10 + (int) $width) . 'px">'
    . do_shortcode( $content ) . '<p class="wp-caption-text">' . $caption . '</p></span>';

    return $output;

}

The above function only changes the opening tags of a captioned image to be <span> instead of <div> (see the last lines). To further optimize the Caption output and make it more readable as well, follow this excellent article by Justin Tadlock.

Now here’s a pointer as to how your CSS should be:

span.wp-caption {
    display: inline-block;
    /* add padding, margin, etc. as needed */
}