Are transients garbage collected?

They now are

Starting with WordPress 3.7 expired transients are deleted on database upgrades, see #20316

Old answer

If someone can’t show me otherwise it seems that transients are not garbage collected after all. What makes it worse is that unlike options they are not guaranteed to be stored in database. So there is no reliable way to fetch list of all transients to check them for expiration.

Some makeshift code to do garbage collection if database is used for storage:

add_action( 'wp_scheduled_delete', 'delete_expired_db_transients' );

function delete_expired_db_transients() {

    global $wpdb, $_wp_using_ext_object_cache;

    if( $_wp_using_ext_object_cache )
        return;

    $time = isset ( $_SERVER['REQUEST_TIME'] ) ? (int)$_SERVER['REQUEST_TIME'] : time() ;
    $expired = $wpdb->get_col( "SELECT option_name FROM {$wpdb->options} WHERE option_name LIKE '_transient_timeout%' AND option_value < {$time};" );

    foreach( $expired as $transient ) {

        $key = str_replace('_transient_timeout_', '', $transient);
        delete_transient($key);
    }
}