If memory serves, the problem is because you are doing a very large query and then running multiple checks across all of them with your max statement.
You might have much better luck breaking it out into two queries so that you can keep the iterative processing to a minimum.
For example, you might:
- Query the posts that have your wanted author
- Query the posts that have the appropriate author term attached
- Combine the results and display them manually.
I don’t have time at the moment to try and come up with a cleaner query … but that query looks a bit processing intensive … and sometimes it really is faster to do two simple queries.