If I use a static page-slug.php for a page, is the database still queried?

Yes. It will. WordPress will query the database for that page regardless of whether or not you use the loop.

If you’re only talking about adding a chunk of content into the editor, then you’re way overestimating the impact of querying the database for a single page.

There are plenty of plugins available that will render your pages to static files that won’t query the database (or run PHP at all), if that’s the direction you want to go in. WP Super Cache and WP Rocket are two that come to mind.