Completely disable categories

With most credit due to this stackexchange answer:

function wpse120418_unregister_categories() {
    register_taxonomy( 'category', array() );
}
add_action( 'init', 'wpse120418_unregister_categories' );

This technically leaves everything in place (so you could bring it back at any time without a loss of data), but it “unhooks” it from everywhere in the admin by removing it from all $object_types.

UPDATE

And to get rid of the Widget, add this to the function above.

unregister_widget( 'WP_Widget_Categories' );