It turns out that it was due to Query Monitor, a plugin that records info about each query. Every time update_post_meta ran, Query Monitor would store some data about the query, which eventually added up to be more than the server could handle.
Running my example code on a default theme like Twenty Nineteen with no other plugins enabled results in the expected behavior:
...
Used memory 1: 5480528
Used memory 2: 5480528
Used memory 1: 5480528
Used memory 2: 5480528
...
So if you’re running into a similar issue, try disabling all of your plugins and switching to a default theme to help narrow down the issue.
update_post_meta
doesn’t appear to be the problem, at least not by itself.