Add caption functionality to custom WordPress theme

When you add an image with caption to the WP text editor, it should add a caption shortcode. Is that the case here?

From there the output html should have some default classes like wp-caption.
You can then style your caption by applying some CSS to these classes:

/* The wrapper <div> for the caption and captioned element. */
.wp-caption { }

/* The caption text. */
.wp-caption-text { }

/* An image within the caption (you might want to style other elements too). */
.wp-caption img { }

See here: http://justintadlock.com/archives/2011/07/01/captions-in-wordpress

Alternatively you can customize the html output using the caption shortcode hook.