Walker gives additional taxonomy name?

Edit: After looking at the wp_list_categories() code I noticed that you can just pass a class parameter to the function. If you pass an empty string, the built-in Walker will always echo class="". To get rid of the class attribute entirely, you will still have to use a custom walker and edit start_el:

function start_el( &$output, $category, $depth = 0, $args = array(), $id = 0 ) {
    // ...
    if ( 'list' == $args['style'] ) {
        $output .= "\t<li";
        if ( !empty($current_category) ) {
            $_current_category = get_term( $current_category, $category->taxonomy );
            if ( $category->term_id == $current_category )
                $class="current"; // removed string operator. not needed here anymore.
            elseif ( $category->term_id == $_current_category->parent )
                $class="current-parent"; // see above
        }
        if ( $class ) // skip output if no class is set
            $output .=  ' class="' . $class . '"';
        $output .= ">$link\n";
    } else {
        $output .= "\t$link<br />\n";
    }
    // ...
}

---

Whenever WordPress runs the callback for start_el it will pass the taxonomy name (plus the default class) as well as other information as 4th parameter: $args.

To remove the default class you could unset/delete the class element in the $args before the extract() line:

function start_el( &$output, $category, $depth = 0, $args = array(), $id = 0 ) {
    unset( $args['class'] );
    extract($args);
    // ...   
}

You would also want to change the code where the walker sets $class to remove the string operator.

if ( $category->term_id == $current_category )
    $class="current";
elseif ( $category->term_id == $_current_category->parent )
    $class="current-parent";