After checking other browsers, I found out that it was a Chrome problem. It looks like it has something to do with Google Chrome notifications. I fixed the problem with adding this CSS to my WordPress admin css file:
/* Chrome media library upload button fix */
@media screen and (-webkit-min-device-pixel-ratio:0) {
.modal-open .rdr-notifier {
display: none !important;
}
}
If someone has a better solution, let me know!