Limit the number of results from wpsc_start_category_query

The wpsc_start_category_query() function basically calls the native WP get_terms() functions that is used to make a wp query:

$category_list = get_terms(‘wpsc_product_category’,’hide_empty=0&parent=”.$category_id);

Since WPEC uses custom post types you can just build your own queries easily, here using http://codex.wordpress.org/Function_Reference/get_terms

But get_terms() doesn”t do random, so you have to get them all and then shuffle the array into a random order, then output the first 4 (any number you set to $max) fo them. So adjusting your layout code it should be:

<ul>
<?php 
//display random sorted list of wpsc product categories
$counter = 0;
$max = 4; //number of categories to display
$terms = get_terms('wpsc_product_category');
shuffle ($terms); //makes list random
if ($terms) {
    foreach($terms as $term) {
        $counter++;
        if ($counter <= $max) { ?>
            <li>
            <div>
                <div class="image"><img src="https://wordpress.stackexchange.com/questions/95659/<?php echo wpsc_category_image($term->term_id); ?>" width="<?php echo get_option('category_image_width'); ?>" height="<?php echo get_option('category_image_height'); ?>" /></div>
                <div class="caption-title transparent_class">
                    <?php echo $term->name; ?>
                </div>
                <div class="caption transparent_class">
                    <a href="<?php get_term_link( $term->slug, 'wpsc_product_category' ); ?>" class="wpsc_category_link"><?php echo $term->name; ?></a>
                    <?php if(get_option('wpsc_category_description')) :?>
                    <?php echo '<div class="wpsc_subcategory">'.$term->description.'</div>'; ?>
                    <?php endif;?>
                </div>

            </div>
            </li>
        <?php }
    }
}
?>
</ul>

I’ve tested the code and works like described.