Set a taxonomy as private

You should also set the query_var to false.
So your args array should be like this:

$args = array(
    'labels'            => $labels,
    'public'            => false,
    'query_var'         => false,
    'rewrite'           => false,
    'hierarchical'      => false,
);

Cheers