Long story short, build a new array with the categories indexed by ID. Then you can just loop over the date-sorted IDs and do with them as you wish:
$categories = get_categories();
$cats_index =
$cats_dates = array();
foreach( $categories as $category ) {
// Index the categories by their ID
$cats_index[ $category->term_id ] = $category;
$posts = get_posts(
array(
'ignore_sticky_posts' => true,
'posts_per_page' => 1,
'category__in' => array( $category->term_id ),
)
);
if ( $posts )
$cats_dates[ $category->term_id ] = ( int ) mysql2date( 'U', $posts[0]->post_date_gmt );
else
$cats_dates[ $category->term_id ] = 0;
}
arsort( $cats_dates );
foreach ( array_keys( $cats_dates ) as $cat_id ) {
// Loop over category IDs by date descending
$category = $cats_index[ $cat_id ];
}