I believe renaming the plugin file/folder should be enough; if not, you can also edit the plugin info in the header of the plugin file and set the version to something unreasonably high. On the rare occasion I’ve needed to do this, I usually also update the plugin info to indicate that the plugin has been modified from the original (and why, if it can be quickly summarized).
The function that performs the update check against the WP.org API is wp_update_plugins. It doesn’t appear to have any filters available to remove a plugin from being sent to the API server, but it is storing the results in a transient called update_plugins so I imagine you could hook into the pre_set_site_transient_{$transient_name} filter to remove your plugin from the list, which should eliminate the admin notification.