I think I found the issue. It looks WPML, the translation plugin, makes a duplicate page for every translated version of a page. So we had multiples of a few pages that were translated into various languages.
Sidenote: WPML stopped working a couple months ago and none of their fixes have worked yet. So no big loss just yet.
Thanks for your help!