My issue was with nginx, specifically:
location / {
index index.php index.html;
try_files $uri $uri/ /index.php;
}
Which I changed try_files $uri $uri/ /index.php; to try_files $uri $uri/ /index.php?$args; as outlined here which also fixed another issue that I was having with URL parameters.