Your SQL isn’t limiting the term matching to categories only. You’re also getting back tags and any other taxonomy. So the data coming back may be more than expected.
Taxonomy is a key in the term_taxonomy table, so it should be fast to eliminate the ones you don’t need.
SELECT DISTINCT(terms.term_id) as ID, terms.name
FROM $wpdb->posts as posts
LEFT JOIN $wpdb->term_relationships as relationships ON posts.ID = relationships.object_ID
LEFT JOIN $wpdb->term_taxonomy as tax ON (relationships.term_taxonomy_id = tax.term_taxonomy_id AND tax.taxonomy = "category")
LEFT JOIN $wpdb->terms as terms ON tax.term_id = terms.term_id
WHERE posts.post_author = {$post->post_author}
Might help. Your DISTINCT also probably causes usage of a temp table for sorting and such. You might try using a GROUP BY instead, like this:
SELECT terms.term_id AS ID, terms.name
FROM $wpdb->posts AS posts
LEFT JOIN $wpdb->term_relationships AS relationships ON posts.ID = relationships.object_ID
LEFT JOIN $wpdb->term_taxonomy AS tax ON ( relationships.term_taxonomy_id = tax.term_taxonomy_id
AND tax.taxonomy = "category" )
LEFT JOIN $wpdb->terms AS terms ON tax.term_id = terms.term_id
WHERE posts.post_author = {$post->post_author}
AND terms.term_id IS NOT NULL
GROUP BY terms.term_id
Might give better results. Set ’em both up, run ’em through phpMyAdmin and have it do an EXPLAIN on them, see which is giving better results.
Also, post_author is an int. You don’t need quotes around it.