Unpublish all posts in a category

You need to get the published posts in the category then run wp_update_post() inserting the new status. Here is a quick function. I am attaching it to admin init. I would also turn it off after your statuses are updated.

function wpse_55018_get_update() {
   for( $i = 0; $i < 10; $i++ ) {     // Runs this loop 10 times
       $args = array(
          'post_status' => 'published',
          'cat'         => 23,        // Use Category ID here.
          'posts_per_page' => 100,    // Lets just do 100 at a time so we don't blow up your server
          'no_found_rows'  => true,   // We don't need to paginate.
          'update_post_meta_cache' => false,  // We don't need post meta either. 
   );
    $posts = get_posts( $args );
    if ( !$posts ) return;             // Breaks out of the function if there are no posts.
        foreach ( $posts as $post ) {
        print_r( wpse_54018_change_status( $post->ID ) );

        }
    }
}

 functionwpse_54018_change_status( $post_id ) {
    $updated_post = array();
    $updated_post['ID'] = (int) $post_id;
    $updated_post['post_status'] = 'draft';

    wp_update_post( $updated_post );
        return $post_id .' Has been Updated';
    }


add_action( 'admin_init', 'wpse55018_get_update' );  This will run the next an admin page is loaded.