When I’m in post editor and click on ‘Add media’, why won’t it load the media library with this code?

The correct hook to add/remove menus in admin area is admin_menu. Also, be aware that current_user_can is intented to check user capabilities, not user roles. See current_user_can documentation and notes.

add_action( 'admin_menu', 'my_remove_menu_pages' );
function my_remove_menu_pages() {

    $user = wp_get_current_user();

    if( ! empty( $user ) && in_array( "contributor", (array) $user->roles ) ) {

        remove_menu_page('tools.php'); // Tools
        remove_menu_page('edit-comments.php'); // Comments
        remove_menu_page('edit.php'); // posts? 
        remove_menu_page( 'profile.php'); // profile
        remove_menu_page('index.php');  
        remove_menu_page( 'post-new' );

    }

    remove_menu_page('users.php');
    remove_menu_page('tools.php');
    remove_menu_page('themes.php'); //appearance
    remove_menu_page( 'edit-comments.php' );

}