Setting cookies in WP REST API requests

Adding this line to my $.ajax call fixed the problem for me.

$.ajax({
    xhrFields: { withCredentials: true },
    // the rest...

Sidenote: this requires the following header to be set on the server-side, which is enabled by default with the REST API (it seems).

Access-Control-Allow-Credentials:true