Scheduling doesn’t work due to caching?

wp_schedule_event is only run when your site is visited and the scheduled time has past. If varnish is serving a non expired cached page then your visitor is not hitting WordPress.

Schedules a hook which will be executed by the WordPress actions core
on a specific interval, specified by you. The action will trigger when
someone visits your WordPress site, if the scheduled time has passed.
See the Plugin API for a list of hooks.

Eventually when your cache expires or is invalidated your wp_cron should fire when varnish rebuilds the cache.