I figured it out. I had the default theme installed with these plugins, Thrive Architect and Ultimate Members. I raised a support ticket with Thrive Architect forums to which I received a reply the reply that when Ultimate Member plugin was disabled everything was working fine.
I further dig into that and came across a stupid configuration error on my part here Dashboard > Ultimate Members > Settings > Access (tab) > Global Site Access I had selected Site Accessible to logged in members only and set the redirect to home page.
I fixed it and it started working properly.