If you need to show the averages in the content, you need to pre-calculate them (before showing the comments).
My approach would be having a custom meta in the post with the calculated averages and modify those metas every time a new comment (rating) is saved.
Something like
add_action("comment_post", "wpse16733_updateAVGs");
function wpse16733_updateAVGs($comment_ID, $approved){
if ($approved){
$commentdata=get_comment($comment_ID, ARRAY_A);
$parent_post=get_post($commentdata['comment_post_ID']);
(... get your rating, get post meta, calc and save ...)
}
}