Under which circumstances is the .maintenance file not deleted?

If you look inside of /wp-admin/includes/update-core.php there is some good info within a comment around line 826.

Here are the steps the upgrader should take…

  1. Test unzipped location for select files to ensure that unzipped worked.
  2. Create the .maintenance file in current WordPress base.
  3. Copy new WordPress directory over old WordPress files.
  4. Upgrade WordPress to new version.
    1. Copy all files/folders other than wp-content
    2. Copy any language files to WP_LANG_DIR (which may differ from WP_CONTENT_DIR
    3. Copy any new bundled themes/plugins to their respective locations
  5. Delete new WordPress directory path.
  6. Delete .maintenance file.
  7. Remove old files.
  8. Delete ‘update_core’ option.

And to answer your question “Under which circumstances is the file not deleted?”

There are several areas of failure. For instance if PHP times out
before step 6, then you will not be able to access any portion of your
site. Also, since the upgrade will not continue where it left off, you
will not be able to automatically remove old files and remove the
‘update_core’ option.

If the copy of the new WordPress over the old fails, then the worse is
that the new WordPress directory will remain.