I have run into the same issue that using $wp_query->set_404();
would properly adjust the global object but not return the 404 template. So in the plugin I was writing I used the tried and tested method:
add_filter( 'template_include', 'wp_139917_force_404' );
function wp_139917_force_404(){
global $wp_query;
$wp_query->set_404();
status_header(404);
include get_404_template();
exit;
}
~ modified from source: https://github.com/codearachnid/woocommerce-product-permalink/blob/master/inc/product-permalinks.php#L56
I have alse used this other format which I believe is cleaner and where appropriate should be leveraged:
add_filter( 'template_include', 'wp_139917_sanity_force_404' );
function wp_139917_sanity_force_404( $template ){
// use your own sanity check logic to return the 404 template
if( your_sanity_check_true_404() ) {
global $wp_query;
$wp_query->set_404();
return get_404_template();
} else {
return $template;
}
}