This isn’t really a WP question and it’ll likely get closed…anyway…your registrar should allow you to do an immediate redirect (called domain forwarding), which will take care if it via DNS. So, keep the domain but shut off the hosting service since you don’t need it.
Updated:
The domain forwarding is a simple HTTP 301 response the registrar provides. You’d need a second redirect at the new domain to get the incoming page to land where you want it to. So for example:
Incoming: www.olddomain.com/some-special-page
Registrar 301: www.newdomain.com/some-special-page
Now, at newdomain: Redirect 301 /some-special-page /my-new-page
Google does not penalize for 301 redirects so I would always try to do it via .htaccess. If the plugin claims it’s “SEO safe” and it is re-writing the response header then that’s probably ok, too.