This seems off-topic, but I’ll answer anyway.
It entirely depends on your arrangement with the client, as well as whether you want to take care of supporting your client’s site after you deliver it to them.
Reputable hosts like wpengine.com provide different levels of access via developer roles: your client pays for everything and has access to everything, then invites you to their account and you have access to all the technical and support tickets functionality needed to do your work (but not the invoicing).
P.S.: Not affiliated with Wpengine, I just know for a fact they provide what I described above.