I want to remove the links from the term list returned by get_the_term_list

You could use get_the_terms() and wp_sprintf_l():

function wpse_52878_term_list( $args = array() )
{
    $default = array (
        'id'               => get_the_ID(),
        'taxonomy'         => 'post_tag',
        'before'           => '',
        'after'            => '',
    );

    $options = array_merge( $default, $args );
    $terms   = get_the_terms( $options['id'], $options['taxonomy'] );
    $list    = array();

    foreach ( $terms as $term )
    {
        $list[] = $term->name;
    }
    return $options['before'] . wp_sprintf_l( '%l', $list ) . $options['after'];
}

echo wpse_52878_term_list( array ( 'id' => get_the_ID(), 'taxonomy' => 'portfolio_cats' ) );

Another option:

echo wp_strip_all_tags( 
    get_the_term_list( get_the_ID(), 'portfolio_cats', ' ', ' , ', ' ') 
);