[Box Backup] Two backup-servers
Markus Liljergren
boxbackup@boxbackup.org
Sun, 27 Jan 2008 17:57:01 +0100
This is a multi-part message in MIME format.
--------------050803030004060804010409
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Hi Chris,
Chris Wilson wrote:
> Hi Markus,
>
> On Sun, 27 Jan 2008, Markus Liljergren wrote:
>
>
>> I'm trying to figure out the best way to deploy BoxBackup as a service
>> for my servers. All services provided by my servers have redundancy in
>> the way that there is always another server ready to take over one or
>> more services in the event of one server failing to provide its assigned
>> services. When implementing BoxBackup in this setup I would like to have
>> all data stored on two different servers (with limited bandwidth between
>> them, for example NFS-based solutions would not work well).
>>
>
> It should be possible to synchronise the two servers with rsync. However
> rsync is not _that_ bandwidth-efficient or disk-efficient. You might
> consider running filesystems that support snapshots at both ends, and then
> locally computing the difference between two snapshots on the master and
> transferring just that difference to the slave. ZFS has built-in support
> for this (using the "zfs send" command).
>
>
The idea of using a filesystem which support snapshots seems very
interesting, though I don't know if that is possible without making your
own port of for example ZFS since I'm running OpenBSD 4.2. Do you know
if this functionality could be achieved using something which comes with
OpenBSD core or any packages?
>> It would be preferable if the clients didn't have to backup to the two
>> servers separately.
>>
>
> I believe that someone here (I think it was Pete Jalajas) was running a
> similar setup and discovered that the bandwidth use between the two server
> sites (for synchronisation) was too high, so he found it better to run two
> completely independent backup configurations on each client.
>
>
If I can't find a solution using snapshots I think the option of using
two independent backup configurations on each client is better.
>> The question I have, is it possible to do it in this or a similar way
>> without risk of corruption due to incomplete syncs on server-failure?
>> Should I just abandon this and have the clients send backups to two
>> different servers?
>>
>
> I think that it is possible and that people are currently doing it.
> I don't believe you have anything to worry about from a failed sync.
>
> If you rsync from server A to server B while clients are backing up to
> server A, you will probably get some incomplete updates, lock files and
> inconsistent store statistics on B. You might want to run "bbstoreaccounts
> check fix" on each account on B after the transfer is complete.
>
> If you do have any problems, please let us know. Good luck!
>
>
Thank you for your thoughts and information. I'm sure I will find a good
solution to this problem.
Regards,
Markus
--------------050803030004060804010409
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Chris,<br>
<br>
Chris Wilson wrote:
<blockquote cite="mid:Pine.LNX.4.64.0801271556230.26095@top.qwarx.com"
type="cite">
<pre wrap="">Hi Markus,
On Sun, 27 Jan 2008, Markus Liljergren wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I'm trying to figure out the best way to deploy BoxBackup as a service
for my servers. All services provided by my servers have redundancy in
the way that there is always another server ready to take over one or
more services in the event of one server failing to provide its assigned
services. When implementing BoxBackup in this setup I would like to have
all data stored on two different servers (with limited bandwidth between
them, for example NFS-based solutions would not work well).
</pre>
</blockquote>
<pre wrap=""><!---->
It should be possible to synchronise the two servers with rsync. However
rsync is not _that_ bandwidth-efficient or disk-efficient. You might
consider running filesystems that support snapshots at both ends, and then
locally computing the difference between two snapshots on the master and
transferring just that difference to the slave. ZFS has built-in support
for this (using the "zfs send" command).
</pre>
</blockquote>
The idea of using a filesystem which support snapshots seems very
interesting, though I don't know if that is possible without making
your own port of for example ZFS since I'm running OpenBSD 4.2. Do you
know if this functionality could be achieved using something which
comes with OpenBSD core or any packages?<br>
<blockquote cite="mid:Pine.LNX.4.64.0801271556230.26095@top.qwarx.com"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">It would be preferable if the clients didn't have to backup to the two
servers separately.
</pre>
</blockquote>
<pre wrap=""><!---->
I believe that someone here (I think it was Pete Jalajas) was running a
similar setup and discovered that the bandwidth use between the two server
sites (for synchronisation) was too high, so he found it better to run two
completely independent backup configurations on each client.
</pre>
</blockquote>
If I can't find a solution using snapshots I think the option of using
two independent backup configurations on each client is better.<br>
<blockquote cite="mid:Pine.LNX.4.64.0801271556230.26095@top.qwarx.com"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">The question I have, is it possible to do it in this or a similar way
without risk of corruption due to incomplete syncs on server-failure?
Should I just abandon this and have the clients send backups to two
different servers?
</pre>
</blockquote>
<pre wrap=""><!---->
I think that it is possible and that people are currently doing it.
I don't believe you have anything to worry about from a failed sync.
If you rsync from server A to server B while clients are backing up to
server A, you will probably get some incomplete updates, lock files and
inconsistent store statistics on B. You might want to run "bbstoreaccounts
check fix" on each account on B after the transfer is complete.
If you do have any problems, please let us know. Good luck!
</pre>
</blockquote>
Thank you for your thoughts and information. I'm sure I will find a
good solution to this problem.<br>
<br>
Regards,<br>
Markus<br>
</body>
</html>
--------------050803030004060804010409--