[Box Backup-dev] Soft-RAID support

[David Sommerseth]

Chris Wilson wrote:
>> "The server currently supports a kind of RAID 5 in userland for extra
>> reliability. It is designed to use three separate paths which are
>> mounted from three separate physical disks (not partitions on the same
>> disk!). This is deprecated and will be removed in a future version. We
>> recommend that you disable it instead, otherwise you may lose your
>> stored data when this feature is removed. "
>> http://www.boxbackup.org/trac/wiki/ConfiguringAServer
>>
>> Is there any reasons this will be changed?
> 
> Support for it was never finished (no recovery procedure), it is pretty
> limited (only supports RAID 5 and three devices) and it was written at a
> time when OS/software and hardware RAID were not as ubiquitous or well
> supported as they are now.

I would be willing, with some guidance to look into such a tool, if that
is the main criteria for dropping this support.

The soft-raid solution itself seems to work flawlessly and seems to only
need this recovery tool.  Or are there any other issues which is not to
well known with the soft-raid which should make me worried?   Are there
any critical bugs related to the current implementation?

> I can see your point about the usefulness of this for distributed
> encrypted backup. However I'm not convinced about the overall merits of
> storing the data in three separate locations. It's already encrypted to
> the point where a server compromise could get virtually no useful
> information out of the backups. You could achieve what you want with
> distributed OS-level RAID on iSCSI, ATA over Ethernet or NBD devices.

Regarding encryption, yes, that is one key element.  But if the
organisation looses one remote storage with the complete backup directory,
it got all the needed information needed to begin to crack the encryption.
 If you need minimum 2 sets to be able to crack the encryption, you have
another layer of security.  And it was this combination which caught my
attention.  When you add locally encrypted disks, you have the third layer
of security.

In general, to achieve the best security, you need as many layers on top
of each other as possible (within your acceptable performance limits, of
course).  And when you need to have 2 separate datasets to make the backup
data readable and useful, it is a very good security layer which you are
about to remove, compared to just have data and storage partition encrypted.

With encryption, you are always dependent on the progress of the CPU
power.  What was considered to be a good encryption 3 years ago, is not as
good today. because it's much easier to crack due to increased CPU power.
 That's what why spreading the encrypted data is just as important as well.

Regarding iSCSI, ATAoE or NBD, that will require more bandwidth.  Those
remote sites I was about to setup, will not have the capacity on the
connection to have such setup working efficient.  By using rsync between
the master and the slaves, the transfer goes much more efficient.

>> I evaluated BoxBackup and set it up before this part of the
>> documentation changed.  Anyhow, there's also a contradictory sentence
>> later on in the same URL:
>>
>> "NOTE Running the server in non-RAID mode has not been tested as
>> extensively as in RAID file mode."
> 
> Strictly speaking, in my mind, this is not contradictory as it doesn't
> say that userland RAID is better or recommended, just more tested.

Yes, exactly.  And that was also why I choose to setup the soft-raid
solution.  Increased possibilities for security, and better tested.

> However I think it may no longer be true. I suspect that few people are
> using the userland RAID feature in production. If anyone except David
> is, please speak up!

I would also be interested in hearing others experiences as well!  If I'm
the only one, I agree, it's not much point in continuing this support in
BoxBackup.  Then I would need figure out another way how to solve this.  I
will not continue on this path if soft-raid disappears for sure in BoxBackup.



kind regards,

David Sommerseth