If your current plugin version is 1.2.5, and you have a beta version for the next version, it should be called 1.2.6-beta. Your user can install it, and when the real 1.2.6 version is released on the repository, WordPress will notify the user on the Plugins page and let him update.
WordPress uses a PHP function called version_compare to compare version numbers for this purpose. When comparing version numbers, WordPress will recognize them like this:
1.2.5 < 1.2.6-beta < 1.2.6
I’ve tested this out with a plugin of my own in the repository, currently at version 0.45.11. I changed the version number of the plugin on my own site to 0.45.11-beta, and WordPress let me update to 0.45.11 from the Plugins admin page.