[Box Backup] Win32 port (was: BoxBackup Server Side Management Specs (Draft0.01))

[Ben Summers]

On 7 Oct 2004, at 22:18, Chris Wilson wrote:

> Hi Ben,
>
>> Yes, you're probably right. I was just thinking about a previous plan
>> for this to allow users to write scripts to provide whatever logging
>> they wanted. Different application here.
>
> OK, cool. For that application it would be easy enough, given the
> existence of a unix socket, to write a small tool that would read its
> output and write it to standard out, which could be executed and read 
> in
> by a shell script. Or alternatively, that functionality could be built
> into Box independently.

Agreed.

>
>>> * state of each file when last scanned (not changed, too new, synced,
>>>   error, excluded)
>>
>> Is this to be queried at any time? Such state is not kept.
>
> OK, I thinkit would suffice to only report the status of files at the 
> time
> that they're scanned. If one wants to diagnose a problem with box, 
> either
> leave the GUI running, connected to the daemon, or connect and then 
> start
> a sync. That way the memory bloat stays with the GUI client, rather 
> than
> the daemon. I think I actually prefer that :-)

Yes, that seems reasonable. I don't want to put lots of code in the 
daemon for things which it doesn't really need to do.

>
>>>  estimated number
>>>   remaining)
>>
>> Not so easy. You don't really know where you are in the backup process
>> until it's finished. It all depends on how many files there are, how
>> much data has changed, etc. Giving a reliable progress indication 
>> isn't
>> an easy task.
>
> It would be easier if we had a copy of the previous state in memory, 
> but
> at least we know, or could find out, how many files we sent to the 
> server
> last time. Then:
>
> 			number of files scanned so far
> current progress ~= 	------------------------------
> 			number of new files uploaded + total last time
>
> which is a good enough approximation for me.

That's easy enough to do, and doesn't require too much extra code.

>
>>> * store status (bytes used, etc.)
>>
>> You can also query the store for this.
>
> I could, but I'd rather not keep a connection open permanently to the
> store, tying up its resources, when I could get the same information 
> from
> bbackupd whenever I need it (or record it for future reference whenever
> bbackupd requests it from the server).

It's only provided on login, or when a specific command is sent to 
retrieve it in more detail.

>
>>> How about XML for the protocol? Here is my first suggestion for the
>>> event types:
> [...]
>> Let's not use XML. I think a simple text based protocol will be
>> sufficient. XML is not the answer to every problem.
>
> I would like to politely disagree with you; XML was designed to be the
> answer to every communication problem. Whether it achieves that goal 
> is a
> different matter, but if not then I'd say it's closer than anything 
> else
> I've seen.
>
> Can you suggest another protocol which is easy to read by humans, easy 
> to
> parse automatically, and fully extensible, and isn't a simple
> transformation of XML? (such as s/<//g; s/>/\n/g).

How about a simple \n terminated terminated line, with tab separated 
message name and arguments? Nice and easy to parse, nice and easy to 
write, extend by adding new messages or arguments.

>
> Also, the write-only implementation for bbackupd would be extremely
> trivial and lightweight. The burden is in the client, which is my 
> problem
> :-)

Why should there be a burden anyway, for the sake of using the latest 
buzzword?

>
>> It's not quite security information. As I understand it, the source 
>> URL
>> and a "zone identifier" is stored in a separate stream, so it knows
>> where the exe came from and whether to run it without warning the user
>> or not.
>
> I agree wholeheartedly that we should have support for streams in Box, 
> but
> I don't yet have time to offer my help with implementing them, sorry 
> :-(
>
>> I must update that list sometime. I have a separate file of notes on
>> things I want to do.
>
> Please do, I would be very interested to see it.

I'll clean it up and post it at some point.

>
> I'm currently adding support for a one-way status socket to my copy of
> bbackupd. I will probably use an XML protocol to start with, although 
> in
> the end I will abide by whatever you deem necessary in order to get the
> code integrated with Box. But I don't promise not to argue hard for my
> side :-)

No XML will not be integrated into the backup engine, sorry.

It might be worth sending me an early copy to make sure the design fits 
in with everything else. Thanks for your efforts!

Ben