Getting users by specific capability, not role

I gave it a test run. First, I added a custom capability to a specific user, using following code

$user = new WP_User( $user_id );
$user->add_cap( 'team_member' );

Luckily, this user can be retrieved using below code, as you suggested

$args = array(
    'role' => 'team_member'
); 

get_users( $args );

Which implicitly work like below

$args = array (
    'meta_query' => array(
        array(
            'key'     => 'wp_capabilities',
            'value'   => '%team_member%',
            'compare' => 'LIKE'
        ),
    )
);

$wp_user_query = new WP_User_Query( $args );

Reference: WP_User_Query