As of now rtMedia Gallery shortcode rtmedia_gallery, doesn’t support count as it’s attribute.
In order to limit the number of media, you have to change the media count from rtMedia Settings. Here is the screenshot:
You just need to set a desired value for media items in the input.
Also you can directly ask rtMedia related questions on the support forum for plugin, it’s quite active.