Modify WP_Query using pre_get_posts but only for frontend query?

Using is_admin() is not hacky, as pointed out in the comments.

You should definitely check for is_admin() when modifying frontend queries. If you’re modifying the main query, also use the is_main_query() function / method.