Remove quicktag buttons but not Visual / Text editor and tabs

As you noted, setting quicktags to false removes the “visual” and “text” tabs. So, to leave the tabs you need to set quicktags to true and remove the buttons:

$settings = array(
    'quicktags' => array(
                       'buttons' => ','
                    )
);
wp_editor($input, 'editor_name', $settings);

To have this in all quicktags instances you can use quicktags_settings filter:

add_filter('quicktags_settings', 'cyb_quicktags_settings');
function cyb_quicktags_settings( $qtInit  ) {
    //Set to emtpy string, empty array or false won't work. It must be set to ","
    $qtInit['buttons'] = ',';
    return $qtInit;
}

If you are using a plugin that add custom quicktags, you may to set a high priority argument to the filter (later execution):

add_filter('quicktags_settings', 'cyb_quicktags_settings', 100);