Can the new REST API now make WP into it’s own push notification server?

Those technologies are totally independent of each other. The rest API can not be used as it is (in this context I mean the end points) for push notifications as there is a protocol for that and probably all kinds of restrictions browsers place. On the other end, nothing was preventing you from implementing those protocols before there was a rest API, and nothing prevents you from doing it now.

there are even plugins that claim to implement it for years by now https://wordpress.org/plugins/push-notifications-for-wp/ (no idea about the quality of it, was just the first result on google)