Avoid repeated post on page 2 when excluding one on page 1

You’ve got single quotes inside your parameter string – use an object instead for clarity:

$.ajax({
    data: {
        action: "infinite_scroll",
        cat: "<?php echo $cat_id ?>",
        exclude: "<?php the_ID() ?>",
        page_no: pageNumber,
        loop_file: "loop"
    }
});