This is serious problem with WordPress and composer. Whatever you do, you are just asking for future problems and/or making life hard for your plugin users. Guess what – even if you manage to load correct version of framework for all your plugins, WP user may unknowingly install some other plugin, which also uses the same framework, but completely different version. And it could get loaded first and be incompatible with your plugins. Those plugins may even not use the same framework, just some other composer libraries, but most of composer libs require lot of other dependencies, and those may easily be conflicting versions.
It’s hypothetical and probability of this actually happening might be quite low, just because most of plugins avoid using composer. But this is also reason why.
To better understand problem, read this and also linked/mentioned articles.
My best advice, if you absolutely need to use framework library, just move it to your own namespace using tools like php-scoper. This way each of your plugins will use their own isolated framework (and their dependencies) versions and will be safe from conflicts with any other plugins using composer.