Not sure what to do next to optimize

It might be useful to analyze where the bottleneck is in rendering pages. Use the Developer mode of your browser (usually via F12) and then use the Network tab to see the load times of the various parts of the page.

Maybe the problem is in images, or ‘off-site’ Javascript code, or any other off-site item that is included in pages. Maybe it is an inefficient template, or perhaps a plugin causing the problem. Looking at page load times will help.

There are also plugins that will show you all kinds of info about database query times. One is described here: http://www.wpbeginner.com/plugins/how-to-add-a-wordpress-query-monitor-on-your-site/ .

Looking at page load times might give you a direction to start concentrating on. If it is a database problem, a query analysis plugin might help.