Display X images from a gallery in the sidebar

Sounds like you need a custom query in order to pull X amount of images. You could do the following (untested):

<?php

function custom_gallery_display( $number_of_images = 4 ) {

    //make sure you have access to the WPDB object
    global $wpdb;

    //pull 4 random attachments
    $images = $wpdb->get_results("SELECT ID from wp_posts WHERE post_type="attachment" ORDER BY RAND() LIMIT $number_of_images");

    //may need to tweak this;
    //not sure what format the results are returned
    $image_ids = implode( ',', $images );

    //now print with random images
    echo do_shortcode('');

    return;

}

?>

That should do the trick, although it may need a bit of tweaking.

You should also check out the documentation http://codex.wordpress.org/Gallery_Shortcode

According to it, you can easily create the randomized order by setting the orderby attribute to RAND

<?php echo do_shortcode(''); ?>

However, if you are forced to use a custom query in order to use X amount of images, you may as well just randomize it at that point, as I did above in the query.