The function you are looking for is wp_set_post_terms()
which can create the terms if they don’t exists and you can specify if to appended or replace existing terms. so you can remove all none user terms.
as for
if post does not have the taxonomy term , add it
you can use wp_get_post_terms() to check if the post has any terms of that taxonomy and if not then you can create and add using wp_set_post_terms() once again