Hide user group filtering links

In WP terminology these are called Views. They are pretty generic, implemented on WP_List_Table level and passed through dynamic views_{$this->screen->id} filter.

Killing them completely for your requirements is as simple as:

add_filter( 'views_users', '__return_empty_array' );

Of course you might want to do something more thorough, like modifying them more precisely and leaving some necessary views in.