I’m familiar with attachment_fields_to_edit and don’t recall ever seeing anything on being able to do modification as granular as you’re suggesting. There’s a WP global $screen (or is it $screens) that perhaps might help but I’m not too sure.
Have you considered – as hacky as it might sound – doing these things with CSS and/or javascript?