[Box Backup] Win32 port (was: BoxBackup Server Side Management Specs (Draft0.01))

[Chris Wilson]

Hi Ben,

> > However, my longer-term plan is to communicate with bbackupd, both by
> > modifying its config file, and using its existing socket, so that it
> > becomes possible to see what the daemon is doing in real time (e.g. why
> > hasn't it backed up File X)
> 
> I was thinking of adding a mechanism to do this would would open an 
> external executable, and pipe in detailed descriptions of what it was 
> up to.

Why an external executable? I can see no reason to prefer this to a
simple unix socket. It adds extra complexity and inefficiency in the case
of a simple client like the one I'm writing. Why would I want to have to
write a program that just stores everything it gets from Box, and spits it
back out over a unix socket? Why not just extend the existing socket
protocol, or create a new one?

> Wouldn't it be better to rewrite the .conf file and send a -HUP signal 
> to do this? Certainly easier.

OK, thanks for the advice, I will try to go that way. 

> I welcome contributions to the code, obviously subject to quality and 
> complexity considerations. I would suggest a quick email with proposed 
> changes to me beforehand to check the design, though!

Good idea. Here goes:

My goal is for a client, connected (by pipe or socket) to bbackupd, to be 
able to recreate all the important parts of bbackupd's state, especially:

* current state in state machine (initialising, idle, syncing, error,
  limit exceeded), time of next sync
* state of server connection (connected/not connected, reason for last 
  disconnection)
* state of each file when last scanned (not changed, too new, synced, 
  error, excluded)
* progress during sync (current file ID, number scanned, estimated number 
  remaining)
* store status (bytes used, etc.)
* statistics on last sync

Ideally, I would like to know when any of these things changes, in order 
to provide live notifications to the user when something is wrong, and 
display live status as they are investigating the reason for a problem, or 
just checking the status of their backup.

I would like to suggest a protocol where, on connection from a client,
bbackupd reports the current value of each of these state items (including
its entire file list) to the client, and subsequently reports the new
value if any of them changes, or the client disconnects and reconnects.

How about XML for the protocol? Here is my first suggestion for the event 
types:

<box:client-status-protocol version="1.0">
	<box:daemon state="init|idle|sync|error|overlimit" 
		nextsync="..." />
	<box:conn state="connected|disconnected" reason="..." />
	<box:file id="[box-id]" parent="[parent-id]" name="..."
		age="..." state="unchanged|toonew|synced|error|excluded"
		errorcode="X/X" />
	<box:progress id="..." count="..." remaining="..." />
	<box:store used="..." old="..." deleted="..." dirs="..."
		softlimit="..." hardlimit="..." />
	<box:stats uploaded="..." unchanged="..." encoded="..."
		sent="..." rcvd="..." time="..." />
</box:client-status-protocol reason="daemon-shutdown|admin-killed">

> Yes, that would be a perfectly acceptable way to do that. It would be 
> better to adjust things so that the in-memory state is retained over 
> restarts to accommodate that without having to query the entire store 
> against the server after every device removal.
> 
> An alternative would be to allow bbackupctl to selectively enable and 
> disable locations.

I think the latter option would be better, especially if adding a new
removable device to the configuration does not require Box to requery the
entire store.

> This use is, but multiple streams in a single file is something NTFS 
> has done for years.

Sorry, yes, I do know about that. But in past versions, the security 
information is intrinsic to an object, not stored in a (user-visible) 
stream?

> There is other work which is higher up on the list, mainly to make it 
> easier to use.

Perhaps I can help you there :-) I had a look at TODO/txt and couldn't see 
any items that appear to relate to ease of use. What needs doing on that 
front?

> As long as I don't lose subscribers unnecessarily, I'm happy.

Me too, I have no desire to inconvenience or annoy you or the other users 
of Box :-)

Cheers, Chris.
-- 
_ ___ __     _
 / __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |