I think your paradigm is not optimal. You are starting on premise of “WordPress Ajax endpoint is slow” and your solution is to “build alternate authentication scheme”. This completely spells trouble.
Anything security should be reused as much as possible and coded by people specializing in security. Trade-off of reimplementing security for the sake of performance is major red flag.
So back to square one. If your issue is “WordPress Ajax endpoint is slow” then build a faster WordPress Ajax endpoint. You can do so with SHORTINIT (there are answers about it around on site) to have very customized core load. It’s a nightmare to ship in public code and pain on upgrades, but for private high–performance Ajax it’s the way to go.
PS I am not sure how your Ajax needs relate to RPC (XML RPC?) needs, since you mention both.