Slow WordPress admin panel with a total query time of 1664ms

What host service are you using? I had the same issue using Dreamhost, where you can adjust the memory limit for the server you are using on the hosts panel. Doing this solved the issue for us. Also, a possible issue could be your SQL server, switching to another SQL server may resolve your issue. Contact your host for more info.