Display the featured image from the last post

This should work. I’ve modified your $args a little and surrounded your echoed code within the while(have_posts()) : the_post(); loop, then added the the_post_thumbnail() function and, finally, the wp_query_reset(); at the bottom:

<?php
    $limit      = 999;
    $counter    = 0;
    $categories = get_categories();

    foreach ( $categories as $category ):
        if ( $counter < $limit ) {
            $args  = array(
                'posts_per_page' => 1,
                'cat' => $category->cat_ID,
                'ignore_sticky_posts' => 1
            );
            $posts = get_posts( $args );

            while( have_posts() ) : the_post();
                echo '<div class="category">';
                the_post_thumbnail();
                echo '<a href="' . get_category_link($category->term_id) . '" title="' . sprintf(__("View all posts in %s"), $category->name) . '" ' . '>';
                echo '<h3>' . $category->name . '</h3>';
                echo '</a>';
                echo '</div>';
            endwhile;

        }

    $counter++;
    endforeach;

    wp_query_reset();

?>

Ps. caller_get_posts is deprecated since version 3.1! Use ignore_sticky_posts instead in your $args