Where to store custom tagging data associated with a custom post type

by

Avoid serialized whenever you can. It is slow to read and you cannot really search for it.

Use custom tables – more than one:

  1. One for the relationship between users, tags and images: user_id, tag_id and image_id.
  2. One for tag meta data, your attributes, if I understand that right: tag_id, meta_1, meta_2 …

Now you can look up for each attachment ID if there is an entry for the user, and then look up the meta data from the meta table.