Sharing Variables between scheduled events

In a literal sense? No, PHP instances are separate, else we’d be discussing multithreading

The problem you face is more fundamental than WP, the only way you’ll be able to keep this in memory is via object caching, which will let you put a copy of the data structure in easy reach.But it won’t save memory, accessing the data will create a copy, and if your machine is not disk IO limited then the performance gains will be minimal

Basically, you would use WP Cache to store the data in the object cache, then make sure there’s an object cache dropin for Redis/Memcached/etc

Using WP Cache functions without this, will keep the data in memory until the end of the request. Using plugins that use file based caching will give you no benefit, and you’ll need to install Redis/Memcached/etc at a server level