Add html link in functions.php files of the theme

You are missing a closing tag for the opening link (<a>) tag, try:

function filter_woocommerce_checkout_must_be_logged_in_message( $message ) {
    $message="<span>You must be <a href="#">logged in</a></span>";
    return $message; 
}
add_filter( 'woocommerce_checkout_must_be_logged_in_message', 'filter_woocommerce_checkout_must_be_logged_in_message', 10, 1 );

(You could also just spit out the login form right there instead of linking off, if you wanted):

function filter_woocommerce_checkout_must_be_logged_in_message( $message ) {
        $login_form = wp_login_form( [ 'echo' => false ] );
        $message="<p>You must be logged in:</p>";
        $message .= $login_form;
        return $message; 
    }
    add_filter( 'woocommerce_checkout_must_be_logged_in_message', 'filter_woocommerce_checkout_must_be_logged_in_message', 10, 1 );