Should is_active_sidebar() always be used around dynamic_sidebar()?

No, but if you’re adding extra markup around the dynamic_sidebar call such as a container, it might look odd if there are no widgets.

Test it out and if it works for you then it should be fine