[Box Backup] exception BackupStore InvalidBackupStoreFilename (4/3)

[Chris Wilson]

Hi James,

On Mon, 5 May 2008, Chris Wilson wrote:

> Out of curiosity, roughly how many "object is unattached" messages did 
> you get in the check run (without fix), and how long is it taking to 
> produce each one when fixing? (this should give us an estimate of how 
> long the fix will take at the current rate).

Sorry, I missed that part in your previous message where you answered one 
of these. If every single object in that range is missing a parent then 
it's about 120,000 objects, of whicit's finished fixing about 5,000 so 
far, in just over a day. That tells me that it's taking about 16 seconds 
each, and it's about 5% finished, so it will take 20-30 days to finish :-(

I think the main problem is this code in BackupStoreCheck2.cpp:

	// The easiest way to do this is to verify it again. Not such a 
	// bad penalty, because this really shouldn't be done very often.
                                                        
	std::auto_ptr<RaidFileRead> file(RaidFileRead::Open(mDiscSetNumber, 
		filename));
	BackupStoreFile::VerifyEncodedFileFormat(*file, &diffFromObjectID);

Unfortunately this is done for every unattached file, in your case 
120,000 of them. That's not compatible with my definition of "not very 
often".

I think it means reading the entire contents of the file in each case. I 
don't know how many files you have in total, or what the encrypted 
filesystem is that you're running on, but I guess it's conceiveable that 
it could take days to read 20 GB of data, especially if seeks are 
involved.

What sort of performance do you get, e.g. how long does it take to read a 
1GB file from this filesystem? Is it CPU or I/O bound? If it's CPU, then 
you might find it much faster to copy the data off the encrypted partition 
before running check fix on it.

Also, since Box Backup's backups are already encrypted by the client, why
put it on an encrypted filesystem?

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\ _/_/_/_//_/___/ | We are GNU : free your mind & your software |