Look at the section on $wpdb->get_results(), it allows you to send distinct queries to the database, which is what it sounds like you are looking to do.
So…
// 1st Method - Declaring $wpdb as global and using it to execute an SQL query statement that returns a PHP object
global $wpdb;
$results = $wpdb->get_results( 'SELECT meta_key, count(meta_key) as votes FROM wp_postmeta WHERE meta_value=" . $user_id . " GROUP BY meta_key' );
This should return an array called $results for the user with the following data:
meta_key | votes
-----------------
vote_up | 15
vote_down| 3
If you want to use a standard function, your other option would be to get all the relevant post_ids, and use get_post_meta( $post_id, $key )
then organize the data in PHP, but the above is more efficient.