I used nextgen gallery plugin to do something like this. Actually I’ve used only half of that: the end user uploads images through the plugin’s interface but the actual display inside the post is done with a custom shortcode.
I don’t think there is a way to tell if an attachment has been inserted in the post, short of examining the post content source.