In Full Site Editing, how do I get the templates I created to appear in the site editor when populating a new page?

When creating a page template in Appearance > Editor you can add a Post Content block. This is where the page content goes. Everything else is part of the template and applies to all Pages that use the template.

When you go to Pages > All Pages and select a page to edit, or use Edit Page in the admin bar, it will open the block editor for editing the page’s content. From here you are only editing the contents of the Post Content block for that page.

Editing the template is the equivalent of editing page.php for classic themes, and the Post Content block is the equivalent of the_content() as used by classic themes.

However, while editing a page, if you edit the template by clicking Page > Summary > Template > Edit template it will open a view where you can edit the page template and the page content simultaneously. This will allow you to see the page content in the page template. It’s important to note that any changes you make outside of the Post Content block will be saved to the template, meaning that they will apply to all pages. Changes made inside the Post Content block will be saved for the current page. If you make changes to both you will be asked which changes you’d like to save when you press Update.

If you want to create a set of blocks that acts as a starting point for the Post Content of a page then the feature you’re looking for is Block Templates.