How to override WooCommerce template files

You should create it. The way woocommerce overriding templates work is that it tries to look for the template in your theme, and when not found falls back to it’s own.

So if you create the exact file structure the woocommerce expects, you’ll be able to override their templates.

I would also consider to try and not do it, and use filters instead if possible, since filters are a much more flexible and future proof solution. When woocommerce will update, the template can change, but you’ll be still with your old one. In case of filters on the other hand, the filter will still work, and you’ll have the advantage of the new template.