So the first thing I would do (in case you have not already) is look at the page source when you are logged out – see if it’s just those “gibberish characters” or if there’s a beginning of html being sent to the browser.
Note that the “gibberish” likely means something – so it might help if you showed us what you get. (or better yet, provided a url to the site)
And then if that doesn’t give you enough of a hint to track down the culprit, I’m afraid you are back down the the good old method of deactivating plugins one by one, until you identify the culprit. And if all your plugins are deactivated, and the problem still happens, then try switching to a WordPress theme like Twenty-seventeen. If the problem goes away, you know it’s your regular theme that is causing the problem.
This should at least point you to a culprit – and then either it’s something you control and can debug, or it’s a plugin you need to give up until the developer fixes the problem. (would be nice to contact them about the issue, too)
Hope this helps?