[Box Backup] Client Processing?

[Ben Summers]

On 11 Apr 2005, at 04:30, Chris Hane wrote:

>
> After using boxbackup for a while now I really like the program.  I 
> have a couple of questions/suggestions.
>
> Once the initial backup has been completed, I am noticing quite a bit 
> of communication between the client and server.  It appears that the 
> client is requesting a directory listing for every directory.  At 
> least that is what I'm guessing based on these entries in the server 
> box.log:
>
> Apr 10 22:12:33 hutz bbstored[3419]: Receive 
> ListDirectory(0x6c,0xffffffff,0xc,true)
> Apr 10 22:12:33 hutz bbstored[3419]: Receive 
> ListDirectory(0x6c,0xffffffff,0xc,true)
> Apr 10 22:12:33 hutz bbstored[3419]: Send Success(0x6c)
> Apr 10 22:12:33 hutz bbstored[3419]: Send Success(0x6c)
> Apr 10 22:12:33 hutz bbstored[3419]: Sending stream, size 89
> Apr 10 22:12:33 hutz bbstored[3419]: Receive 
> ListDirectory(0x6d,0xffffffff,0xc,true)
> Apr 10 22:12:33 hutz bbstored[3419]: Receive 
> ListDirectory(0x6d,0xffffffff,0xc,true)
> Apr 10 22:12:33 hutz bbstored[3419]: Send Success(0x6d)
> Apr 10 22:12:33 hutz bbstored[3419]: Send Success(0x6d)
> Apr 10 22:12:33 hutz bbstored[3419]: Sending stream, size 89
>
> Question: Would it be possible to streamline this communication (or 
> maybe even maintain some state on the client).

A lot of state is maintained on the client. It's just that it's 
maintained in memory. When you just leave it running, you'll find it 
makes very few requests to the server.


>   Right now with my test backup machine (e.g., my laptop), I am seeing 
> long connection times.

I assume you shutdown your laptop quite a lot. This causes the bbackupd 
daemon to lose the state -- I designed it originally to work on servers 
which were just sitting there.

Of course, in my planned redesign to take into account all the comments 
and feature requests, state will be cached to disc to avoid this 
problem.


>  I have around 20,000 files in hundreds of directories and just to do 
> an incremental backup takes hours (each directory listing is being 
> downloaded and compared?) even though very few files have been 
> modified.  As a note: I'm running the client on Windows XP in lazy 
> mode with a default config setup (except of course for my 
> path/account/key info)
>
> Second question, I have noticed that if there is a connection error 
> between client/server that the client keeps retrying once every minute 
> or so to reestablish the connection.  Our configuration is a bit 
> unique in that during the day I have to limit our bandwidth 
> consumption for non-client services and hence turn off the backup 
> server (just shutdown the process).  Is it possible to configure the 
> retry connection?  Or maybe even have the time between retries gradual 
> extend (say 1 minute increments for 5 minutes, then once every 5 
> minutes for an hour than once an hour - just a suggestion).

Not in the current version. Although I can point you at the bit of code 
if you'd like to patch it yourself?

>
> Third, what is the best way to shutdown the backup server.  I 
> currently send a signal for the command line; but it seems that 
> established connections will not shutdown until the current backup it 
> is performing is completed.  This is how I even noticed my first 
> question....

Send a kill signal to all processes. The server uses a transactioned 
and failure tolerant system to write data to the store, so you can't 
leave it in a dodgy state if you just randomly kill it off.

Again, this will be fixed.

>
> Also, I now the windows port is being worked on separately from the 
> unix client/server.  Has it been integrated/released yet?  Although 
> it's been a while since I've hacked on C code.....

Not yet. Too many time constraints.

Ben