How to customize categories panel?

First you have to find the right action to hook into. I find it by look into class-wp-posts-list-table.php the columns is in an array so you just have to find the right key and unset it by php-function unset.

Here is a example that removes all of the columns:

function wpse_111473_remove_category_columns( $defaults ) 
{
  unset( $defaults['description'] );    // Description
  unset( $defaults['slug'] );           // Permalink
  unset( $defaults['posts'] );          // Posts count

  return $defaults;
}
add_filter( 'manage_edit-category_columns', 'wpse_111473_remove_category_columns' );