Any extra code on your server is a theoretical security risk. Instead of checking each plugin and theme if they are security risk even if they are not active, it is much easier to just delete them.
This of course should not prevent you from backuping them first in case you will decide that you need them in the future 😉