You would use the handle of your enqueued scripts, ie, the first parameter of the wp_enqueue_script function used in WordPress for all your scripts… (if you are loading them another way you should use this function instead.)
The handle for jQuery is… jqueryjquery-core now. But each script queued will have it’s own handle. A bit like a ‘slug’ for a post.
You are fine to add as many handles to that array without having to change the filter parameters. (The 10, 2 part is the filter run priority and number of arguments passed to the filter function respectively.)
But the script_loader_tag filters enqueued scripts, not stylesheets, so no this will not be fine for doing the same with CSS files.
EDIT
I don’t think that loop will work as it is, as it will return on the first fail, and thus not check all the handles. Use instead:
foreach($scripts_to_async as $async_script) {
if ($async_script === $handle) {return str_replace(' src', ' async="async" src', $tag);}
}