Super confusing ‘pre_get_posts’ behavior with $query->set

You don’t need to add a pre_get_posts filter for ?tag=xyz to work, URLs in WordPress should already work that way. If you can write it as a string and pass it to WP_Query then you can append it to a URL with a ? to add additional query parameters. That is assuming that the main query hasn’t been replaced with a new one, or that other plugins haven’t interfered

What’s really happening with $query->set()?

Before a query executes it’s passed through the pre_get_posts filter, and set changes the query variables. That’s it, there’s no strange voodoo going on

Why does it require me to set any random stuff in $query->(blah, blah) in order to work properly?

It shouldn’t, but when the front page settings are involved things can get a little strange, as it changes what is_homeand is_front_page mean. On the main query this means that is_front_page can’t be used because the logic that figures it out hasn’t run yet

What if I want to search by /search/?topic=someTopic instead of /search/?tag=someTopic?

Do what you’ve been doing but instead of $_GET['tag'] do $_GET['topic']

A Final Note on Your Template

I see this in your question:

return plugin_dir_path( __FILE__ ) . 'partials/display-speaker-list.php';

That template will need a standard post loop for any of this to work. If any of the following are true, then it will not:

• A query_posts call
• A WP_Query loop
• Using get_posts

None of those are the main query, so the pre_get_posts filter will fail the check and move on without making the modification

As an aside, have you considered changing the tag permalinks instead and using real archive URLs instead of hacking the main blog page with URL parameters?