Why does WordPress allow inserting tags into posts?

It is my understanding that there should not be more than one <h1> tag on a page, and that the <h1> tag should be the site title, for SEO reasons.

But that’s your opinion, not a rule. From an accessibility perspective the h1 should be used for main content’s title, on a single page that is the post title, on a list (archive), the archive title. Otherwise, a user exploring the content per headline hierarchy has no idea what this page is about. Screen reader users do that very often.

WordPress is not made for some obscure SEO voodoo (sorry ;)), it is for real people. These are different, they have different needs and different opinions. And that is the reason why you can use any headline level on any page.

Also, would it be a good idea for theme designers to not style <h1> tags to discourage users from using them?

Users would just edit the theme file, create a child theme or use a better theme. Not a good idea. You could filter the content on save_post and turn h1 elements into h2 elements. I would not do that.

The elements WordPress does not allow per default are mostly potential security problems: embeds and scripts. And even that can be overridden.