Ofcourse! It always loads and runs the code in the main plugin file, that’s by design and is how plugins are intended to work. Why would it work any other way? How would it know without being told which code to run when?
There are no sandboxes that ensure your classes only work in certain scenarios, they’re still written in the PHP programming language. Computers are very literal, it always runs because you never told it not to run in certain situations.
WordPress will load your activated plugins and your themes functions.php files, which are just PHP files that get executed. Anything you do in those files will run, wether it’s a frontend request, admin area, REST API, or even an RSS feed. It will run and any that is not in an action will immediately run, and it isn’t sandboxed or cordoned off from the rest of WordPress or the other activated plugins. It’s all loaded into the same memory at the same time in a single shared space.
So What Should I do?
What you need to do, is use actions to schedule when things are meant to happen. If you want to specify that something should only happen in a certain situation, you have to implement that yourself via hooks and if statements. Doing things in the main plugin files outside of actions other than adding actions and filters or loading more PHP files is ill-advised.
For example, you should only register admin pages when in WP Admin, so you would hook into the admin_init event and do work there, ensuring the code does not run on the frontend and other locations. Likewise you would register REST API endpoints for AJAX on rest_init. This also improves performance.
The rare exception is uninstall.php. If no uninstallation hook is registered it will load uninstall.php when a plugin is uninstalled. This is because you can’t register an uninstallation hook if the plugin is not activated.