I have used symbolic links for convenience for awhile on a dev server but they are not 100% reliable as some PHP functions will return the filesystem path of the target instead of the path to the symlink itself. The most notable example (though not really a function) is the magic constant __FILE__. (Also https://bugs.php.net/bug.php?id=46260) That can cause trouble, and it can be frustrating to sort out. For that reason I am moving away from symlinks even on the dev server. Perhaps with some combination of this and related functions you can make this work reliably, but I don’t think I would.
If you are distributing this plugin publicly you have the additional problem of creating those symlinks on the various servers where the code needs to run.
Your files should not be any less accessible at site/wp-content/plugins/plugin/assets/ than site/wp-content/uploads/plugin/assets if you get the permissions right, but I’d make them accessible over some other mechanism, like the AJAX API, if it were me.