I created and managed my own custom meta fields for years and I can tell you it was always a headache. I frequently came up against stumbling blocks like yours and ended up banging my fist against the wall. I’ve also worked on quite a few sites which had been flooded with plugins, sometimes so many that it soured me on plugins altogether and I resolved to use them only when absolutely critical to a project.
One of the only reliable plugins I came across was Advanced Custom Fields. I’ve used it on almost every project since first trying it out and it’s truly expanded the possibilities of all my projects. It really organizes fields quite nicely and for $25 they also have a repeatable field add-on which I’ve become quite familiar with.
Give them a shot.