[Box Backup] RAID - software instead of hardware?

[richard_eigenmann]

My thoughts:

I'm not running Software or Hardware RAID. BoxBackup is my backup tool. If it
breaks I will simply back up again. Of course it would be highly inconvenient
if the backup breaks at the same time as the source also runs into trouble.
In my case I have my data replicated exactly between the main workstation and
my Laptop (using unison) giving me a first level of protection.

If we are thinking of the 5 year timeframe then the key element is storage
explosion. In 1999 I bought 10 and 25 GB disks and they were pretty big then.
Now you pick up 200GB for half the price.

If / when your BoxBackup disk breaks then you will likely pick up a new one
which is magnitudes larger than what you had. This will not fit well into the
two other software or hardware raid partitions / devices. Chances are you
will then want to rebalance the size allocations and replace the other
devices too.

If run without software RAID BoxBackup simply uses a directory structure.
This you can move easily to a larger device as your storage requirements
grow. You have no hassle with repartitioning 3 potentially different capacity
devices. With LVM it should even be possible to use several devices together
to create one large filesystem if you have some old disks lying around. But
then my experience is that old disks come in sizes of 800MB or 4GB which
makes that a bit pointless.

To safeguard against a disk failure on the BoxBackup server I would suggest
getting a second backup server at a separate offsite location. Any clapped
out PC (that doesn't suffer the IDE 34GB BIOS limit) should be able to run a
BoxBackup server on a 200GB disk. 200GB are going round for +/- 130 USD.

I'm wondering if we could even help similar minded BoxBackup users to offer
each other backup storage capacity. Such as Alice and Bob who live on
different continents and have a permanent connection (even if relatively
slow) each set up a clapped out PC with 200GB storage in the basement and
then back up to their local server and to the other's machine. This implies
network connectivity and the presence of a basement. It also implies a
certain degree of trust between the parties as you are letting a stranger
onto your network. But then people are dating each other via the Internet
which seems to me to involve a higher degree of risk.

Of course if you are running BoxBackup in a large corporate infrastructure
then you probably have Hardware Raid anyway. I have my doubts about how
suitable BoxBackup is in that sort of environment though. How do you comply
with data retention requirements? How do you retrieve that fraudster's .pst
file from 3 years ago to get at that incriminating email? With tape that is
possible by keeping the yearly, monthly and possibly weekly tape sets locked
away in a vault for the retention period. BoxBackup would have to have some
sort of configurable minimum retention period definition for the housekeeping
process. Since this is likely to be 5 or 10 years it implies that we need
incremental updates in the backup vault if we want to have any chance of
surviving such a volume of data. Also we might have different retention
requirements for different classes of data.

Regards,
Richard