yup it is fast, it would be even better for you to use the WP Transients API to cache your query for a short period.
Basically:
if($saved = get_site_transient('SOME_SORT_OF_ID')){
echo $saved;
} else {
ob_start();
switch_to_blog(x); // x being the blog id your want to switch to
// Do your quering and echoing here
restore_current_blog(); // Revert to current blog
set_site_transient("SOME_SORT_OF_ID", ob_get_contents(), (60 * 60)); //Save query for one hour
ob_end_flush(); // Echo and End Buffering
}