Only allow a plugin to work on a specific page? (Prefer not to use a plugin)

Yes, there is a way to do this as described in this blog post. Due to the way WordPress loads files, you must use it in a Must Use Plugin and not the theme’s functions.php or a regular plugin.

I’m not the author of that blog post and don’t want to copy the script from there. The idea is to add a filter to option_active_plugins in which you unset() the plugin from the array you are filtering.