From a WordPress perspective, there’s not a lot to configure for performance. You can set define( 'WP_DEBUG', false ); in your wp-config.php to disable logging, which will gain you some minimal savings.
Beyond that, I would recommend working on making sure your heavier plugins are optimized as best as you can, reduce the number of requests if possible, and optimize your queries.
You can take a look at plugins like Debug Bar to get an idea of what may be taking up the most resources on any given request, and services like New Relic can provide deeper insights to what’s going on with your server in general.