How to backup 20+TB of data?

You need to take a step back and stop thinking “I’ve got 20TB on my NAS I need to back up!” and develop a storage strategy that takes into account the nature of your data:

  • Where is it coming from and how much new data are you getting? (you’ve got this in your question)
  • How is the data used once you have it? Are people editing the pictures? Do you keep the originals and generate edited versions?
  • How long do you need to keep all the data? Are people still making changes to pictures from 2 years ago?

Depending on the answers to the last two questions, you probably need more of a Archiving System than a radically different backup system.

Data that is static (e.g. 2 year old pictures that you retain “just in case”) doesn’t need to be backed up every night, or even every week, it needs to be archived. What you actually do might be more complex, but conceptually, all the old pictures can be written off to tape (multiple copies!) and not backed up any more.

Based on your comments, some additional thoughts:

  • Since you keep the originals of each shoot untouched and work on a copy, and assuming that at least some of the original pictures are duds, you might be able to cut the amount of data that needs to be backed up in half.

  • If you still can’t finish a full backup within whatever window of time you have, a common way to speed things up is to do a disk-to-disk backup first and then later copy the backup set off to tape.

Leave a Comment