Is there a way to use ‘publicly_queryable’ => false only on specific taxonomy term?

ok – so this isn’t what I wanted but it does the trick.
Adding a function to redirect when the user hits any term but one:

function my_page_template_redirect() {
    if( is_singular( 'custom-post-type' ) && !(has_term('term', 'custom-taxonomy')) ) {
        wp_redirect( home_url(), 301 );
        exit();
    }
}
add_action( 'template_redirect', 'my_page_template_redirect' );

Answer based on this one: Disable Single Post View for Specific Taxonomy on Custom Post Type