[Box Backup-dev] COMMIT r476 - box/chris/general/bin/bbackupd

[Charles Lecklider]

Chris Wilson wrote:
> Why is an extra thread wasteful? I know that synchronisation would be
> easier without it, and multithreaded programming is always tricky, but I
> thought that threads were lightweight, and this one (now) spends almost
> all its time in WaitForMultipleObjects.

Threads are indeed lightweight, but there's no point in having more of
them than you need.

My point wasn't that an extra thread isn't needed at all, just that
*all* IO should be handled from it, not just the control pipe.

>> Chris: If you're intent on learning overlapped IO you want to be looking
>> at completion ports. They're essentially a message queue, but without
>> all that tedious messing around with critical sections etc. You can wait
>> on them from multiple threads, and the wait has a timeout (which is what
>> I was planning on using for SSL keepalives - if no completion packet
>> comes in after X ms send the keepalive - no timer required).
> 
> OK, thanks, I will have a look at them. What API functions are involved?

Start with CreateIoCompletionPort, then GetQueuedCompletionStatus. It's
very simple once you get your head around it, but a little hard at first.

>> However, once you start looking at it you'll soon realise that there's a
>> lot more that needs to change in order to take advantage of completion
>> ports properly. The good news is that none of the actual box code needs
>> to change (or at least I didn't see any in my quick review), and the new
>> code will be simpler in a lot of ways than the old, but the bad news is
>> that pretty much the entire Win32 architecture needs to change.
>>
>> That's why I wasn't planning on working on it until 0.10 is done - start
>> adding this now and there'll never be a release....
> 
> I'm not sure if it will go into 0.10, but this BADLY needs fixing. The
> command socket is so dangerous as to be almost unusable on win32.

I have no plans to use the 0.10 Win32 release because there are too many
things that need fixed before I would feel comfortable putting it on a
client's machine. The *nix version is a good step forward, and IMHO
needs to be released ASAP.

Once there's a 0.10 I plan to start working on the Win32 version, as in
actual development, not just these emails ;-)  I'm not confident with
branching in SVN, and the merge functionality looks primitive at best,
so I need a fixed point to start from. (I'm sure it's actually truly
wonderful and the best thing since whatever, but I don't have time to
find out).

So because of all that, my view on 0.10 is simple: as long as Box
compiles clean with a set of VC2005 project files, sod the Win32 bugs,
ship the release. If we do that, then 0.11 (or whatever the next will be
called) will have proper security and all sorts of overlapped IO
goodness in it, and will be suitable for general use, because I *need*
that PDQ.

-C