Make a really simple gallery structure

Use your own gallery shortcode handler – something like this in your functions.php:

function __my_gallery_shortcode( $attr )
{
     // render the gallery the way you want it
}
add_shortcode( 'gallery', '__my_gallery_shortcode' );

To get you started, you could just copy the code from the default handler gallery_shortcode(), and edit as you require.