Where do I remove admin order fields (unset doesn’t work)

Maybe try this:

function custom_admin_billing_fields( &$billing_fields ) {
   unset($billing_fields['state']);
}

It’s not clear if when the custom_admin_billing_fields function is called if its return is actually captured and used. So it’s possible that the return is ignored.

The ampersand (&) in the parameter definition makes that variable “by reference” instead of “by value”. This way any changes made to this variable in the function will always be returned and affect the original variable passed.