How can transfer get so large when exporting data (15-16MB to over 10GB)

You seem to be confusing different things. I am not sure what precisely you mean by “transfer” (monthly bandwidth? disk space?), but 10GB figure likely isn’t about memory (RAM).

Error you are getting is about RAM. 100663296 bytes corresponds to just 100MB. That much is typically enough for average WP install to function, but might not be enough for some specific functionality or third party code. I think the highest WP tries to request natively is 256 megabytes when manipulating images.

You should inquire with your hosting about RAM available and configuring it to higher value. If that is highest you can get you might not be able to do resource–intensive operations.