Eventually after confirming that the database had been updated I rebooted the server I tried moving the file wp-admin/upgrade.php to be wp-admin/411.upgrade.php. This got the site out of its death spiral.
Given that I had manually updated all the files using Upgrading_WordPress_Extended and had also managed to get the database upgrade installer to run as shown by the db_version being 30133 entry in the _options table if you run into the same problem trying moving that script out of the way.