Redirecting “wrong” but working links to their canonical URL in order to prevent search engine penalties (e.g. /123/ at the end of URLs)

Ensure that you provide a canonical url – this should ensure that search engines don’t see the url variants as duplicate content.

Additionally, appending a number on the url, will make WordPress try to access that page.

Using a debug plugin, such as Debug Bar, is quite useful for diagnosing this kind of behaviour as it will show the query performed on each page.