The best way to tell if any given remote service is alive is to ask it to service a request in the way it’s meant to – in fact it’s the only way to truly know something is working correctly.
As an example I always get my load-balancers to get an actual ‘head’ response back from our web servers, you could do the same for a small select on a DB box if you wanted to, or whatever your actual server serves. As a tip you can create an ‘online.txt’ (or whatever name you want to give it) on your web servers, have your LBs try to get that file and if it fails then it removes the server from the VIP, this is a nice way of manually taking individual servers out of your VIPs simply by renaming a single file.
Ping only tests for the ability to respond to pings, so that’s base OS, parts of the IP stack and the physical links – but that’s all, everything else could be down and you’d not know.
I know this is mentioned below, but it bears repeating again and again.
ICMP Echo Requests (aka “Pings”) (aka ICMP Type 8) are built onto the IP stack spec, yes, but are not required to be either implemented or used. As a matter of fact, there are a large number of internet providers who refuse to forward those and silently drop those requests, as they are a form of network attack (called a pingflood).
As mentioned above, this is handled by the OS (specifically at the network stack level) and so it is up to the OS configuration to respond to those or not. If this is turned off (a security precaution?), you can’t do anything about receiving ping replies from the other end. This is why it’s not reliable.