Figured it out. Missed the concatenated class dot for the data-filter attribute…
Changed this line:
echo '<button id="category-filter" data-filter="'.$term->slug.'">'.$term->name.'</button>';
To this:
echo '<button id="category-filter" data-filter="' . '.' .$term->slug.'">'.$term->name.'</button>';