Retrieve custom fields on Categories, using WP-API

Turns out it’s the helpfully titled rest_prepare_categoryfilter

function prepare_restful_categories($response, $item, $request) {
    // Do stuff to categories
}
add_filter('rest_prepare_category', 'prepare_restful_categories', 10, 3);