Use of comment_reply_link_args filter

You mean, something like this?

add_filter( 'comment_reply_link_args', 'change_author_title', 10, 2 );
function change_author_title( $args, $comment ) {
    $args['reply_to_text'] = 'Reply to ' . get_comment_author( $comment );
    return $args;
}

Explanation: I changed the function declaration so that it accepts the second parameter ($comment) which is the comment object (a WP_Comment instance), then I simply call the get_comment_author() which fires the get_comment_author hook. That way, your my_comment_author() function would be called and then you’d get the same author name for use with the reply_to_text arg.