You’ll find it much easier to control which fields are shown, by controlling which walker is used. Rather than trying to ‘toggle’ the fields inside the walker.
I also wanted to show a bunch of custom fields, and only show them on a menu at a particular menu location. So I used the following…
add_filter( 'wp_edit_nav_menu_walker', function( $walker, $menu_id ) {
$menu_locations = get_nav_menu_locations();
if ( $menu_locations['main_menu'] == $menu_id ) {
return 'Walker_Nav_Menu_Custom';
} else {
return $walker;
}
}, 10, 2);