How can I conditionally add the filter option_home?

Assuming that if wp_sitemaps_enabled has a filter and it has that because we wanted to set it to false, otherwise it defaults to true for public sites. So I would just use and check whether it is filtered or not:

add_filter( 'wp_sitemaps_enabled', '__return_false' );

if (has_filter( 'wp_sitemaps_enabled') === true) {

    add_filter('option_home', 'any_callback_function');

} else {

    // echo "the Sitemap is not filtered";

}

Other useful approach could be if you use the WP_Sitemaps object to check the status of sitemaps:

add_filter( 'wp_sitemaps_enabled', '__return_false' );

$thisSiteMap = new WP_Sitemaps();

$site_map_status = $thisSiteMap->sitemaps_enabled(); 

// var_dump($site_map_status); // just to check the value

if ($site_map_status === false) { // which means sitemaps are filtered/not available

    echo "sitemaps are turned off";
    add_filter('option_home', 'my_uri_function'); // any custom callback function
}

// example callback function
function my_uri_function() {

    $myurl =  get_site_url() . '/shop';

    return $myurl;
}