Creating custom admin panel pages without making a plugin?

As you have seen, you only need to use the function add_menu_page (),

although the documentation assume that you created a new plugin this is not necesary to do it.

you can create a new file in your theme and use include_once at the end of functions.php or simply add these functions directly into functions.php