Why is my oembed function modifying Twitter embed?

You should be doing some sort of check at the beginning of your function to ensure you’re in an oembed that you actually want to mod. something like

if (strpos($url, youtube.com) === false && strpos($url, vimeo.com) === false) {
    return $cache;
}

Although you might want to elaborate on that to catch shortened urls like youtu.be as well.