Getting users with a specific meta data and then querying their posts?

Note that here: $posts = new WP_Query, the $posts is not an array of posts and instead, it’s an object. So instead of doing the foreach and setup_postdata(), you should just use the while ( $posts->have_posts() ) ... method to display the loop. But then I guessed you originally used $posts = get_posts() there?

I mean, I could enter any non existing meta_key or meta_value and it is still displaying all user posts.

So I guess I have made a mistake somewhere in the code.

No, you did not, and the problem is neither with get_users().

It’s just that when you use a non-existent meta key/value, then that means get_users() will return an empty array which is then passed to WP_Query via the author__in argument, and when it is an empty array, then WP_Query will not limit the query to certain post authors only, so 'author__in' => array() is actually the same as not setting that argument and thus, what you get when it’s not set would be the same as what you’d get when it’s set to an empty array.

Therefore, what you would want to do is, run the new WP_Query only if the $user_ids is not empty, like so:

if ( ! empty( $user_ids ) ) {
    $query = new WP_Query([
        'author__in' => $user_ids,
        // ... your other args.
    ]);

    if ( $query->have_posts() ) {
        while ( $query->have_posts() ) {
            $query->the_post();
            the_title( '<h3>', '</h3>' );
        }

        wp_reset_postdata();
    }
}