Is it Ok to restrict Access-Control-Allow-Origin for /wp-json requests?

First of all, this is intentional behaviour, as relayed in a Slack discussion described in this ticket (this has likely been discussed in other places, but that’s the first I found):

tl;dr: CORS is built for CSRF protection, but WordPress already has a
system for that (nonces), so we “disable” CORS as it gets in the way
of alternative authentication schemes

it’s a design decision to expose data from the REST API to all
origins; you should be able to override in plugins easily

So it should be noted that it’s perfectly normal for wp-json/ to be accessible from all origins, and it’s not inherently insecure.

As for your questions:

Is it okay to set restrict the origin in this way?

By modifying your WordPress install itself? Absolutely not. You should never edit WordPress core files. Any changes you make will be overwritten any time WordPress updates, forcing you to make the change again every time, possibly leading you into the very bad practice of maintaining your own fork of WordPress.

If you want to modify core WordPress behaviour yourself, you need to create a plugin, and use the Hooks API to make your changes by removing and replacing actions, or filtering values.

In this case, you’ll notice that the header is sent in the rest_send_cors_headers() function. This function is run by WordPress by hooking it with this line:

add_filter( 'rest_pre_serve_request', 'rest_send_cors_headers' );

Which, according to inline documentation is inside a function that is:

Attached to the {@see 'rest_api_init'} action to make testing and disabling these filters easier.

So, if you want to modify rest_send_cors_headers() what you need to to is create a plugin, and inside that plugin copy the rest_send_cors_headers() function to a new function with a different name. Then you want to replace WordPress’ version with yours:

add_action(
    'rest_api_init',
    function() {
        remove_filter( 'rest_pre_serve_request', 'rest_send_cors_headers' );
        add_filter( 'rest_pre_serve_request', 'my_new_rest_send_cors_headers' );
    }
);

Where my_new_rest_send_cors_headers() is the name of your new modified version of the function.

What might be the consequences of doing it? Maybe plugins that rely on a header like access-control-allow-origin: * could stop working? Are there such plugins?

Most plugins are unlikely to have issues, as they will be making the requests from the same origin, however you will have issues with two types of plugins that I can think of:

  1. Plugins that work with an external service. That service may want to connect to your website via the REST API, and will be unable to do so if you are only allowing requests from one origin.
  2. Plugins that make requests to your site’s REST API from the server, using curl, or the WordPress HTTP API, without setting the Origin header to your site’s URL. This is pretty unusual and rare, but not impossible.

That last point brings up an important issue: Setting Access-Control-Allow-Origin is not going to protect your data. It is trivial for a server to spoof the origin and make a request to the API and get that data. This header is not an authentication security measure, so if you want to lock down your API from the outside world, this header is not the way to do it.