Is it possible to load a different sidebar in single.php based on a meta_query filter?

First off – it sounds like you might actually be better separating your speaker types by using a taxonomy.

Taxonomies are for grouping things together and the query that WordPress runs, for example: to get conference speaker is faster than doing it by meta query.

Also you have the added bonus of being about to use taxonomy archive which means it’s easier to write conditionals.

So for this example I would have a taxonomy called ‘speaker_type’. You can then use the template taxonomy-speaker_type.php to style the output (this might be roughly the same as you’re using for your post archive archive-speakers.php)

You can then use the conditional in your sidebar:

is_tax( 'speaker_type', 'conference' );

Hope that helps.