Where can I store data in WordPress?

In my experience, update_option() and update_post_meta() are what I need over 90% of the time. Use update_option() if you want to save data related to the site overall, and use update_post_meta() if you want to save data related to a specific post.

The other ~10% of the time I might need something more esoteric, like set_transient() if I want to temporarily save some data (eg if I want to cache something) or wp_insert_term() if I want to add a new term to a taxonomy.