tax_query: Order by slug?

If I understand your scenario, your events are in the right order but your semesters are not.

If you change line 4

$terms = get_terms('semesters');

to

$terms = get_terms('semesters', array(
    'orderby' => 'slug'
) );

that should put the semesters in the order you’re looking for. ‘order’ defaults to ASC, which is what you need to display 14, then 15, then 16, so you shouldn’t need any other parameters.