I know this post is very old but still answering for others who reach here.
Following is a very simple filter to sort the terms as numbers. You only need to set the variable $taxonomy_to_sort
with taxonomy slug
and place it in functions.php
of your theme.
add_filter( 'get_terms_orderby', 'terms_order_as_number', 10, 3);
function terms_order_as_number($order_by, $args, $taxonomies){
$taxonomy_to_sort = "numbers";
if(in_array($taxonomy_to_sort, $taxonomies)){
$order_by .= " * 1";
}
return $order_by;
}