The action is not right after actually. There is get_header() call, then get_header action and then locating and loading template file that has wp_head() in it.
I try not to mess with removing things from inside of same hook you are at, so in this case I’d (ab)use that get_header action to hook function that will remove what you don’t want from later wp_head.