You may be able to have WordPress push data to the session or to your custom PHP, rather than trying to load WordPress each time you need to query some the user. I’m not sure what other functions you want to call.
The alternative is to use the WordPress REST API, take a look at the authentication section.