How to optimize performance without transient?

I don’t think a transient per post is a bad idea at all, and it’s certainly more efficient than firing the API call on every load. Just be sure to give your transient name a unique name – maybe append the post ID to it.