All attachment pages give 404

Actually I got to the bottom of it.

The problem was that I have installed the WPML plugin. But I forgot to install the WPML Media plugin. As soon as I installed it, the attachment pages started working again.

In my opinion, this is unacceptable and I will escalate the issue to WPML team. If the Media plugin is so critical, it should be made obvious to the users or it should be install automatically.