I figured it out! The problem is woocommerce.php file. This file overwrite the other directory. In my case, I’m developing a custom theme that’s why I need to create a directory inside my ../themes/my-theme nameed- /woocommerce/archive-product.php and delete the previous one woocommerce.php form ../themes/my-theme
So, moral of the topic is- If your theme has a woocommerce.php file, you will be unable to override the woocommerce/archive-product.php custom template in your theme, as woocommerce.php has priority over other template files. This is intended to prevent display issues.