Is the “lost password” feature truly a vulnerability?

»The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with armed guards – and even then I have my doubts.« (Gene Spafford)

The lost password feature might potentially be a risk, but if you are keeping it or not is just a matter of risk assessment. For that assessment there surely is not the one answer, you have to assess it for your case yourself. Last but not least, I personally would say, it is not to an extent unsafe, which would justify the suggestion, to always remove the feature. So I wouldn’t say it generally is a serious vulnerability.