In the WP_Admin_Bar::add_menus() class method you will find the actions:
add_action( 'admin_bar_menu', 'wp_admin_bar_my_account_menu', 0 );
add_action( 'admin_bar_menu', 'wp_admin_bar_my_account_item', 7 );
and the two corresponding callbacks are using get_avatar().
To prevent the avatar changes in the admin bar, we can remove the foo_change_avatar filter before these callbacks and then add it again after the admin bar is rendered with the wp_after_admin_bar_render filter (it looks like this filter is not documented in the Codex):
add_action( 'admin_bar_menu', function(){
remove_filter('get_avatar','foo_change_avatar');
},0);
add_action( 'wp_after_admin_bar_render', function(){
add_filter('get_avatar','foo_change_avatar');
});
So this part of the admin menu bar:

where the get_avatar() is used, should be excluded from the changes.
We could also have used the admin_bar_menu filter with priority greater than 7 instead of the wp_after_admin_bar_render filter.