If you look in core, that filter is applied like so:
$image_meta = apply_filters( 'wp_calculate_image_srcset_meta', $image_meta, $size_array, $image_src, $attachment_id );
Instead of using __return_empty_array
you need to add a custom function to that filter to remove the srcset size you are trying to get rid of. Something like this.
function my_srcset_function($image_meta) {
if (!is_array($image_meta)) {
return $image_meta;
}
if (!empty($image_meta['sizes']) && !empty($image_meta['sizes']['sow-carousel-default'])) {
unset($image_meta['sizes']['sow-carousel-default']);
}
return $image_meta;
}