get_terms with posts that have a different taxonomy with term x

Just to start, you say “I am using get_terms to retrieve all terms from Tax 1 with posts associated with it”, however your actual code gets all terms in that taxonomy regardless of whether or not they have posts. You need to set hide_empty to true.

So with that out of the way, the bad news: You won’t be able to do this with get_terms(). That type of information doesn’t exist without querying posts and comparing the terms. So the messy solution you wanted to avoid is unfortunately necessary.

Ultimately I think this is a situation where you might be better off just using SQL. This code will use $wpdb to get terms from Taxonomy A that have any posts of a given post type that have a specific term in Taxonomy B:

$post_type="post";
$taxonomy_a="post_tag";
$taxonomy_b = 'category';
$term_b_id  = 12;

$query = $wpdb->prepare(
    "SELECT DISTINCT
        terms.*
    FROM
        `wp_terms` terms
    INNER JOIN
        `wp_term_taxonomy` tt1 ON
            tt1.term_id = terms.term_id
    INNER JOIN
        `wp_term_relationships` tr1 ON
            tr1.term_taxonomy_id = tt1.term_taxonomy_id
    INNER JOIN
        `wp_posts` p ON
            p.ID = tr1.object_id
    INNER JOIN 
        `wp_term_relationships` tr2 ON
            tr2.object_ID = p.ID
    INNER JOIN 
        `wp_term_taxonomy` tt2 ON
            tt2.term_taxonomy_id = tr2.term_taxonomy_id
    WHERE
        p.post_type = %s AND
        p.post_status="publish" AND
        tt1.taxonomy = %s AND
        tt2.taxonomy = %s AND
        tt2.term_id = %d",
    [
        $post_type,
        $taxonomy_a,
        $taxonomy_b,
        $term_b_id,
    ]
);

$results = $wpdb->get_results( $query );

if ( $results ) {
    $terms = array_map( 'get_term', $results );
}

So that code gets all Tags that have Posts that are assigned to the Category with the ID 12.

The last little bit with get_term() ensures that the final array contains the same WP_Term objects that you would get by running get_terms().