For every scaling question you can look at wordpress.com and see how big they are. I do’t know the details but I think that with all the users of wordpress.com itself, akismet users and jetpack users, an estimate of 1M users can’t be totally wrong.. In addition so far as far as auttomatic people say they are running vanilla wordpress (or very very close to it) with plugins.
So can it scale scale above 100k users? seems like the answer is an easy yes. So how much time does it take them to load the users page? I think the answer is that wordpress do not have the tools to easily admin this amount of info and just because of UX reasons you will never want to go to the users page when you have so many users.
So maybe there are two separate questions here, Can wordpress scale to have 100k user? yes! Does vanilla wordpress give you the tools to easily admin so many users? probably not.