$wp_customize->remove_section for customizer setting?

Assuming you are trying to remove the default nav_menus panel, you have the id wrong. Also, you’ll need to add a priority of at least 20 to the customize_register hook, assuming you’re using that hook.

function remove_customizer_settings( $wp_customize ){

  $wp_customize->remove_panel('nav_menus');

}
add_action( 'customize_register', 'remove_customizer_settings', 20 );