Using $wpdb | checking entered email against existing emails in db

You shouldn’t be using mysql_num_rows() or mysql_error() when dealing with $wpdb. Even if you weren’t, mysql_error() is for database errors, and an empty result set is not a database error.

If you want to know if results were returned, simply check the count() of the results:

$duplicateEmail = $wpdb->get_results($emailTest, OBJECT);

if ( count( $duplicateEmail ) > 0 ) {
    // Email already exits.
}

In terms of ‘returning an error’, that depends entirely on the context. If you just want a blank screen with an error message, then you could just use wp_die():

$duplicateEmail = $wpdb->get_results($emailTest, OBJECT);

if ( count( $duplicateEmail ) > 0 ) {
    wp_die( 'Email already exits.' );
}

Or if this is an AJAX request you could use wp_send_json_error() to return a 400 error code with a message in JSON:

$duplicateEmail = $wpdb->get_results($emailTest, OBJECT);

if ( count( $duplicateEmail ) > 0 ) {
    wp_send_json_error( 'Email already exits.', 400 );
}

Also, you really should not put variables directly into SQL unescaped like you are. This leaves you vulnerable to SQL injection attacks. Instead use $wpdb->prepare() to generate the query including your variable. You should also use $wpdb->prefix so that the query works even if the user is using a database prefix other than wp_:

$query   = $wpdb->prepare( "SELECT * FROM {$wpdb->prefix}email_subscribers WHERE email = %s", $email );
$results = $wpdb->get_results( $query );

if ( count( $results ) > 0 ) {
    wp_send_json_error( 'Email already exits.', 400 );
}