Removing buttons from the html editor

With the quicktag_settings filter:

function wpa_47010( $qtInit ) {
    $qtInit['buttons'] = 'strong,em,link,block,del,img,ul,ol,li,code,more,spell,close,fullscreen';
    return $qtInit;
}
add_filter('quicktags_settings', 'wpa_47010');

The default is:

$qtInit['buttons'] = 'strong,em,link,block,del,ins,img,ul,ol,li,code,more,spell,close';

Though 'fullscreen' usually gets added too at the end. So I just deleted the 'ins' button.

Edit to add:

If you wish to create a custom button the following tutorial might help.