RFC: end-to-end compare -aq (Was: Re: [Box Backup] Win32 native client service bbackupd.conf)

[Ben Summers]

On 7 Jul 2006, at 00:41, Gary wrote:

>>> Would it make sense for the client to download, from the server, a
>>> list of blocks where the information supplied about each block is
>>> (1) the IV used previously by the client to encrypt that block, and
>>> (2) the current checksum on disk (on the server) of that block,
>>> freshly computed by the server?
>
>> Yes, I think that would do the trick nicely.
>
> The server actually stores those IVs? Interesting - what for?

Just to send back to the client later. Knowing the IV is very useful  
when it comes to decrypting the data.

> I presume they are stored on the
> serever in an encrypted state?

No, in the clear. There's no need to encrypt them, they don't give  
any information which is useful to an attacker. All you're saying is  
that the secret information in the first block is XORed with this  
value, which leaves them none the wiser about the value of the  
plaintext. (But of course stops attacks based on comparing cipertext  
blocks.)

>
> I missed that when looking through the sources.

Check out BackupStoreFile::EncodeChunk()

So, basically what you'd need to do is, for each block, send IV,  
encrypted length, unencrypted length, and ciphertext hash. That  
should allow you to encrypt each block in turn with the given IV, do  
the hash on the client, and compare. You will of course need to send  
the block index because the unencrypted length of the block is stored  
in the encrypted section of the block index. (see BackupStoreFileWire.h)

Now there is a slight potential problems with this. Most blocks will  
be compressed, so you are replying on zlib emitting the same output  
every time it compresses the same data. Now this should work, but a  
change in zlib could give different output and screw up this  
comparison process.

We could implement our own compressor. zlib may not be the best  
choice of compression, because it only operates on small blocks.  
Perhaps a more simple compressor could give just as good performance?

>
>> It doesn't require the server to be trusted any more than the minimal
>> "store files and let us retrieve them intact" requirement.
>
> The alternative of the client sending up to the server the  
> ciphertext along with strong checksums
> for the ciphertext, to be stored and compared by the server later  
> on, would also allow for an
> end-to-end bbstoredcheckaccount (without requiring client  
> cooperation).

I'm not sure I quite follow what you're suggesting.

Ben