[Box Backup-dev] Timeouts

[Ben Summers]

On Thu, 7 Dec 2006, Chris Wilson wrote:

> On Thu, 7 Dec 2006, Ben Summers wrote:
>
>
>> Hmmm. A better plan would seem to be redesigning things to cope  
>> better with
>> unreliable connections.
>>
>> 1) Gather all changes to send before connecting in the first place  
>> (from file
>> system notifications or a slow scanning operation)
>>
>
> We can't gather file diffs that way, unless we store the encrypted  
> block
> checksums on the client as well.

Sorry, I meant "files which have been changed and need to be  
uploaded" not doing diffs beforehand.

>
> And if we did, then it might be a multi-gigabyte set of changes  
> that we'd
> have to store until we connect.


Diffs are stored as a 'Recipe' and the data built on the fly, so  
they're not that memory intensive. But yes, not the best plan ever.


>
>
>> 2) Reconnect and continue when the connection fails.
>>
>
> Yes, that would be good. How would we handle that? I guess we can't  
> just
> catch the exception and log in again, but it would be nice to be  
> able to
> detect the progress of the upload (maybe store it in the context  
> object)
> and restart from the same position after a failure.
>
> Perhaps if we pushed the stack of pending directory entries into the
> context at each stage, then we'd be able to easily pick up where we  
> left
> off? Would require a bit of rework to do all the readdir()s first.

If we have a prepared list of files to change, we can avoid the  
recursive code.

>
> What's so bad about the new timers implementation and keepalives  
> that I
> spent a month writing, including full unit tests?

Nothing at all. Avoiding the timeout in the first place is far better  
than anything else! Diffing will happen when the connection is open.

Your work is appreciated!

Ben