Increase search results for Admin -> Appearance -> Menus -> Search (default is 10)

No need to change core files! Here is a hook (add in functions.php or simple plugin):

// filtering quick-menu-search results (this seems better than others at https://pastebin.com/raw/jRkJYAzE )
add_action( 'pre_get_posts', 'myFilter1', 10, 2 );
function myFilter1( $q ) {
    // example of $q properties: https://pastebin.com/raw/YK1uaE0M
    if(isset($_POST['action']) && $_POST['action']=="menu-quick-search" && isset($_POST['menu-settings-column-nonce'])){    
        // other parameters for more refinement: https://pastebin.com/raw/kZ7hwpyx
        if( is_a($q->query_vars['walker'], 'Walker_Nav_Menu_Checklist') ){
            $q->query_vars['posts_per_page'] =  30;
        }
    }
    return $q;
}

change 30 to whatever you want.