Automatically Generated Menu pages

There are two major things you need check for to resolve this. If your site is using the WordPress menus, then under Appearance | Menus is a checkbox that if ticked will automatically add all new top-level pages to the menu. Unticking that box will stop the behavior.

If you check that area and no menus are defined, then the theme is using wp_list_pages() to generate a menu. If the menu bar appears at the top of the content, then check the theme’s header.php file for the function. If your menu is on a sidebar, check for the function in sidebar.php.

If neither of the above applies, you might check your Widgets to see if something there is doing it.