How can I compel alphanumeric entries for username textbox on woocommerce registration form

You can use the woocommerce_registration_errors hook to validate the username & set an error if your conditions aren’t met – WooCommerce will bail and pass the error back to the UI for the user:

add_filter( 'woocommerce_registration_errors', function ( WP_Error $errors, $username ) {
    if ( ! preg_match( '/[a-z]/i', $username ) ) {
        $errors->add( 'username', 'Username must contain alphabetic characters.' );
    }

    return $errors;
}, 10, 2 );