Can the uniqueness of the plugin file/plugin slug and version number be trusted?

Not really. There’s no 100% reliable way to guarantee that wordpress-seo/wp-seo.php on one site is the same plugin as wordpress-seo/wp-seo.php on another site, short of hashing the files, as you mentioned.

It is highly unlikely that they are different plugins, but since there is no centralised system for registering unique plugin names, it cannot be guaranteed with 100% certainty.

Plugins on the wordpress.org repository obviously have unique names, but there’s no stopping a developer using the same slug for a plugin distributed elsewhere. This can actually cause problems, as WordPress may report an update available for an external plugin if there is a plugin with the same name in the wordpress.org repository with a higher version number.