How do I handle SSL properly when WP is behind a reverse proxy?

You can’t hook is_ssl() to override the result, and as you’ve noticed you can’t edit WordPress Core itself or your changes will get lost if you’re using built-in automatic updates.

So the usual approach – see the WordPress documentation – is to set $_SERVER['HTTPS'] = 'on';, which is the property is_ssl() tests. Add the following block to wp-config.php (which is preserved during updates), somewhere before the final require_once:

if (strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false) {
    $_SERVER['HTTPS'] = 'on';
}

This tests whether your reverse proxy added a header X-Forwarded-Proto: https to the proxied request and if it did it sets the HTTPS flag for WordPress, so that the SSL flag does try and reflect the original request. The Really Simple SSL plugin has a more comprehensive version of this that I’ve used too that tests more values from other proxies:

//Begin Really Simple SSL Load balancing fix
if ((isset($_ENV["HTTPS"]) && ("on" == $_ENV["HTTPS"]))
  || (isset($_SERVER["HTTP_X_FORWARDED_SSL"]) && (strpos($_SERVER["HTTP_X_FORWARDED_SSL"], "1") !== false))
  || (isset($_SERVER["HTTP_X_FORWARDED_SSL"]) && (strpos($_SERVER["HTTP_X_FORWARDED_SSL"], "on") !== false))
  || (isset($_SERVER["HTTP_CF_VISITOR"]) && (strpos($_SERVER["HTTP_CF_VISITOR"], "https") !== false))
  || (isset($_SERVER["HTTP_CLOUDFRONT_FORWARDED_PROTO"]) && (strpos($_SERVER["HTTP_CLOUDFRONT_FORWARDED_PROTO"], "https") !== false))
  || (isset($_SERVER["HTTP_X_FORWARDED_PROTO"]) && (strpos($_SERVER["HTTP_X_FORWARDED_PROTO"], "https") !== false))
  || (isset($_SERVER["HTTP_X_PROTO"]) && (strpos($_SERVER["HTTP_X_PROTO"], "SSL") !== false))
) {
  $_SERVER["HTTPS"] = "on";
}
//END Really Simple SSL

And there’s an alternative approach here on StackOverflow where you can use Apache configuration to set HTTPS=1 instead if that’s easier:

<IfModule mod_setenvif.c>
  SetEnvIf X-Forwarded-Proto "^https$" HTTPS
</IfModule>

rather than editing wp-config.