Can I count matches without doing a query?

The non perfect solution is to just minimize the amount of information carried by the query, and the side effects. This can be done by requesting only the post IDs and not populating any caches.

Your query should be something like

$query = new WP_Query(array(
  ...
  'fields' => 'ids',
  'cache_results' => false,
  'update_post_meta_cache' => false,
  'update_post_term_cache' => false,
));

So you still get too much information – an array of integers instead of one, but since the DB has to go over them in any case the main overhead is probably the time to transfer it, and if you do not expect 1000s of posts matching your query, then it should be good enough.

The reason I would prefer this approach over trying to figure out all the required joins and write a proper SQL, is that it is much easier to read and modify than an SQL statement. Still requires a good comment about the why of it.