http_referrer issue when detecting where site comes from and allow if from allowed host

This is from the PHP website. This may be the problem.

‘HTTP_REFERER’

The address of the page (if any) which referred the user agent to the
current page. This is set by the user agent. Not all user agents will
set this, and some provide the ability to modify HTTP_REFERER as a
feature. In short, it cannot really be trusted.