Obtain full v2 REST url?

You could add the path argument to the get_rest_url() function as per the WordPress Developer docs – https://developer.wordpress.org/reference/functions/get_rest_url/ – in order to get the path you need:

get_rest_url( null, 'wp/v2/' );

This would produce: http://mywebsite.com/wp-json/wp/v2/