[Box Backup] Server redundancy and backup servers

[Ben Summers]

On 24 Sep 2004, at 08:10, Ben Summers wrote:

>
> I have been discussing redundancy for servers off-list, and have come 
> up with some plans and preliminary design notes. A copy is below for 
> your comments.
>
Thanks for the comments so far.

It has just occurred to me that using the built in software RAID, a 
limited form of redundant servers could be created. Someone suggested 
this on the list a while back, and I've only just realised the 
implications.

All you need are three identical servers. On each server, compose the 
RAID file sets from the local hard drives and the two hard drives from 
the other servers (mount the discs using NFS or something.)

Run the bbstored daemon on each, and use round-robin DNS with a low TTL 
to send clients to different machines.

It should then "just work". If any machine goes down, then the software 
RAID will kick in and no-one will notice, apart from the administrator 
who will notice the log messages.

The changes required are:

* Add communications between bbstored servers so that a client can log 
in even if another server is housekeeping that account.

* Account database syncing between servers.

* Raid file disc set restoration tools needs to be written (which is 
still currently lacking -- right now you have to move the existing 
files away in case they're needed, then blank every account and wait 
until the clients have uploaded everything again.)

* Efficiency: write the raidfile daemon to offload RAID work, and write 
the temporary files to the local filesystem only.

The advantage over the previous plan is that most of the work is 
already done -- none of the above is a particularly significant amount 
of effort. The disadvantage is that it limits clusters to three 
machines which are connected to each other with fast network 
connections. However, it is a rather neat and simple solution.

Your thoughts, as always, are welcomed.

Ben