Infinite scroll loads new posts several times

The problem indeed was that there were several posts on the bottom. Adding this to the end of the loop helped:

<br style="clear:both;"/>

I think this is a bug in the Jetpack plugin, but it is easily resolved by adding this line break.