How to remove UI access to Custom Post Type using direct links?

Yes. Use capabilities! When registering a post type, take a look at the capability_type and capabilities.

You can define a custom capability map for the standard edit_post, delete_post, etc. for your post type.

The thing you remember is, if you customize the capabilities, YOU DON’T HAVE THESE CAPABILITIES EITHER! This means you’ll have to get those capabilities on your user role. (probably Administrator?)

You need to then either create a new user role and assign capabilities to it, or just use a plugin like Members which gives you a good interface to create these and assign capabilities to existing roles, as well as creating new roles altogether.