It was an issue with W3 Total cache. Since the Database cache was activated for non logged in users only, it was not working as intended anymore.
Now the weird part… The Database cache was activated for months and it always worked well, until I upgraded WP to 3.5.2 (and probably W3TC in the same time).