[Box Backup] Box-Backup slow: 0.1 MiB/sec

[Ben Summers]

On 13 Aug 2006, at 17:58, Magnus Homann wrote:

> Ben Summers wrote:
>> On 9 Aug 2006, at 18:13, Felix E. Klee wrote:
>>> At Wed, 9 Aug 2006 17:23:33 +0200,
>>> Baltasar Cevc wrote:
>>>> Try to check whether the IO on the machine is the problem. My old
>>>> backup server (which is a machine with about 2 GHz) had serious
>>>> trouble with the whole bunch of tiny files bbstored writes.
>>>
>>> Thanks for the hint.  I may indeed try that out.  However, first I'd
>>> like to know whether it's possible to disable encryption of the
>>> transmission channel and, if so, how.
>> It's not possible.
>>> Also, I'd like to know why the
>>> transmission channel is encrypted: After all, the backups  
>>> themselves are
>>> encrypted.  So, additional encryption seems to be superfluous,  
>>> something
>>> for the paranoid.
>> The protocol around the backups isn't encrypted. Using SSL gives you
>> * Strong authentication
>> * Protection from man-in-the-middle attacks
>> * And obscures the stuff being transmitted, reducing the  
>> information available to an eavesdropper to just quantity and timing.
>> The cost is a bit of public key crypto at the beginning of each  
>> session, which would need to be done anyway for authentication,  
>> and symmetric crypto for all data during the session. The latter  
>> is not very processor intensive, and certainly shouldn't take up  
>> 95% of CPU time.
>
> How about optionally being able to turn SSL off, and let the user  
> have the possibility to use SSH tunneling instead?

The reason I used SSL was to avoid the necessity for everyone to have  
a UNIX account at the remote end. This makes handling backups for  
many users much more manageable.

> That is often more available at the remote end (at least for  
> me :-). Now I  do the backup to a local disk, and then do a rsync  
> over ssh to the far end. Probably more I/O efficient, but NOT disk  
> efficient! :-)
>
> I looked at the code, but didn't see an obvious way to modify it.

Write an ssh tunneling IOStream derived class -- see lib/common/ 
IOStream.h and the documentation in the notes directory.

bin/bbackupd/BackupClientContext.cpp,  
BackupClientContext::GetConnection(). Modify to check the daemon  
configuration (via the mrDaemon member variable), and if ssh is  
required create one of your new classes instead of the SSL socket. No  
other modifications are necessary for the client.

The server is slightly more complex. I suggest getting the daemon to  
listen on a local UNIX socket (with appropriate permissions) and a  
helper exe which simply redirects stdin/stdout to this socket. In bin/ 
bbstored/BackupStoreDaemon.cpp check the configuration again, if  
using the ssh option, get the UID of the connecting process, choose  
the account based on that UID (account number == UID?), and just use  
a normal non-SSL socket. The templating of the class is a problem,  
you may need to have some sort of compile time option to choose which  
option. Or move everything to a lib/ directory, and have  
BackupStoreDaemon be templated itself, and have two exes, bbstored  
and bbstored-ssh to select which option you want.

Not too much work, I suppose. :-)

Ben