limit the access to uploaded files

If people can guess the file download urls then they will still be able to get to them (e.g. if someone who has access shares the download urls out).

To get around that kind of problem you typically need to set up a file handler script that will grab them from a protected directory that prohibits direct content downloading and serve them up.

That being said, what you are doing is generally good enough unless it is vitally important that people not be able to find and download things.