As stated in the official blog of jQuery. Note that WordPress is mentioned in the quote.

jQuery Migrate 1.4.1 released, and the path to jQuery 3.0

Version 1.4.1 of the jQuery Migrate plugin has been released. It has only a few changes but the most important of them fixes a problem with unquoted selectors that seems to be very common in some WordPress themes. In most cases Migrate can automatically fix this problem when it is used with jQuery 1.12.x or 2.2.x, although it may not be able to repair some complex selectors. The good news is that all the cases of unquoted selectors reported in WordPress themes appear to be fixable by this version of Migrate!

A quick answer to your question; yes you can remove the jQuery migration script and if you don’t see any unwanted behavior after the removal of the script, then it’s safe to say that you can completely remove the reference migration script.

Can be read in here