How can I remove fields in the attachment editor?

Use the attachment_fields_to_edit filter to remove the fields you don’t want displaying from the array.

function remove_caption($fields) {
  unset($fields['post_excerpt']); // See wp-admin\includes\media.php line 1071
  return $fields;
}
add_filter('attachment_fields_to_edit','remove_caption');