I’m probably overcomplicating things.
Right now, you are.
As I have no traffic as of now I don’t really need to do this, but
…. I don’t know what to choose as I don’t
understand what these services really do, other than it’s supposed to
be “better” to host externally…
Then why bother with a CDN right now and all of the image hosting options you describe?
Get a good web host that can handle the little traffic you’re getting now and with the small images and thumbnails you have, the site will be fast with little effort or complication.
Concentrate on the other and simpler WP performance issues, like an optimized database, the latest version of PHP, raising WP’s memory allocation, browser caching, image optimization with http://wordpress.org/plugins/ewww-image-optimizer/ etc. Those aspects will go much farther right now than a CDN.
For a cache plugin, use WP Super Cache http://wordpress.org/plugins/wp-super-cache/ with PHP caching, which is easier to set up than mo_rewrite.
If and when you need a CDN for traffic, use Super Cache’s CDN integration and read those docs on how to use it and what it will help with. In my experience, Amazon’s CloudFront or the “easy” version of CloudFlare (free at many web hosts) are the easiest CDNs to use ad integrate.
It does get complex: a Content Delivery Network (CDN) is a cache in some respects, but a cache is not necessarily solely a CDN. And CDN is an external host by definition, external to your main host (which can execute code, for one). Dropbox/Google Drive can be caches and CDNs, but I doubt they are optimal in terms of network capacity. All free image hosting – like Flickr, Picasa, etc. – can be a CDN for images, but they have APIs you will have to work with.
But keeping it simple is the best idea for usage and reliability.