Remove image of srcset

If you look in core, that filter is applied like so:

$image_meta = apply_filters( 'wp_calculate_image_srcset_meta', $image_meta, $size_array, $image_src, $attachment_id );

Instead of using __return_empty_array you need to add a custom function to that filter to remove the srcset size you are trying to get rid of. Something like this.

function my_srcset_function($image_meta) {
    if (!is_array($image_meta)) {
        return $image_meta;
    }
    if (!empty($image_meta['sizes']) && !empty($image_meta['sizes']['sow-carousel-default'])) {
        unset($image_meta['sizes']['sow-carousel-default']);
    }
    return $image_meta;
}