Why do I keep losing links when I switch to visual editor in WordPress

The problem is your <div> inside of your anchor tag. See this answer on StackOverflow about what tags are allowed inside of anchors.

Inline elements ( a, span, strong, em among others ) can contain other inline elements and text nodes.

and

Generally, block-level elements may contain inline elements and other block-level elements. Generally, inline elements may contain only data and other inline elements. Inherent in this structural distinction is the idea that block elements create “larger” structures than inline elements.

Anchors are inline, while <div> tags are block-level, hence the block-level element cannot go inside of the inline element. If you need the element inside of the link, try using a <span> with the display:block CSS property:

<a href="somelink"><span class="tag">Link</span></a>