Allowing custom role user to edit post assigned to them but don’t let them create new custom type post

You cannot find the capability because it does not exist. In WP out of the box, the paradigm is not post creation, but post publishing. Users can be adjusted to edit and create drafts and pending posts, but not publish.

Setting users to only edit however is not possible at the roles and capabilities level, and implementing that would be a significant amount of work on WP Core itself. For example, this would break autosaves and revision history if it was possible as is.

Additionally, the internal code paths for update posts vs creating posts are near identical. The important part being wether a post ID is specified when wp_insert_post is called.

It might be possible that you can intercept and prevent the posts creation during on of the post save hooks, but this is likely to have unanticipated knock on effects, as well as the UI still implying post creation is possible.

To summarise:

  • This isn’t possible with vanilla WP
  • WP itself is built under the assumption it isn’t possible
  • Making it possible is a significant amount of work involving WP Core ( possibly months or years of work )
  • You could kludge your way to it with filters, but the results will be awful with lots of difficult non-trivial bugs
  • The REST API and XMLRPC could bypass those

I strongly discourage against further pursuing this path, advise that it will be expensive in the extreme, and that the chances of failure are incredibly high. Find an alternative.