But recent months, several sites of my clients were hacked and I’m concerned about this problem.
In my experience pattern of hacks in quick succession indicates a common link. Typically it is vulnerable plugin/theme or incompetent hosting.
If you do not consider the option of password brootforcing, how hackers can get access to the file system of the site and upload malicious files there?
Hacking is arms race by nature. If there was a way to simply enumerate the ways in which hack can occur then it would be possible to simply close all of those ways. But it’s not.
Is it real protect site without using Security Plugins (e.g. iThemes Security), which create additional load? What would you advise on server-side?
Plenty of developers I know are extremely skeptical of WordPress security plugins. There are a lot of installations out there not using them and working just fine.
Personally the one addition I consider necessary is plugin for two-factor authentication. It has low probability of causing issues and keeps access secure even if credentials were leaked.
Is it real to protect WordPress without updates? Sometimes there is no way to automatically update old projects. Two sites have been hacked: v3.x (which is understandable) and v4.x (up to date)
No, it is not. By nature of software any large project still contains security vulnerabilities to be discovered. For each major release of WordPress there are typically couple minor releases with bugfixes and security fixes.
Recent versions of WordPress started to automatically update for security releases. You could consider opting in to major releases as well, but that comes with increased risk of things breaking without testing.
Overall staying on specific version of WordPress indefinitely is simply bad.