Would it be better to ditch the JS formatter on the input fields in the admin and instead add the dollar sign and comma separation on the front end?
Yes, definitely.
This seems more logical, but then it raises another question of how to
prevent a user entering a dollar sign or comma formatting in the
admin.
Sanitize the input before storing it, using absint().