I hope you are using different theme in each site
Just add this code in your header.php file.
<link rel="shortcut icon" href="https://wordpress.stackexchange.com/questions/25736/<?php bloginfo("stylesheet_directory'); ?>/favicon.ico" />
Add your favicon in each theme.
For favicon in admin area.
Add the following function in your function.php file.
function admin_favicon() { ?>
<link rel="shortcut icon" href="https://wordpress.stackexchange.com/questions/25736/<?php bloginfo("stylesheet_directory'); ?>/favicon.ico" /> <?php }
add_action('admin_head', 'admin_favicon');