Need help deciding on a taxonomy

I’d suggest Apparel Collection as a custom post type. Shirts, Jeans etc as Categories. Colors, size, etc as tags.

That way, any body searching for only XL size can group all apparel with tag XL. You can further group per-apparel along with a tag. Should make your job easier.

Also, if you decide to create another collection, you can just create another custom post type for that as well. It will be cleaner in the back-end too.