[Box Backup-dev] Re: [Box Backup] bbackupd - read errors on database files

[G.]

Martin,

> FWIW I'm undecided on this. I don't know what data-set Gary has run it
> on, but 15% overhead doesn't sound very realistic to me. Bearing in mind

A quad-core QX6700 test server, striping on 10K Raptor drives, around 500GB backup set with 10% to ~25% typical change cycle. You are certainly right that this is a "your mileage will vary" type of situation here, and I was taking only approximate masurements, but that's what I was seeing: no overly significant slow-down (maybe it's because I spend a lot of time in large file diffs anyway). Checksum information already available for comparison in bbackupd in-memory cache and used for folder-level checking made things tolerable. It's easy to check with a simple ad-hoc piece of code before committing to an implementation.

> Clearly file notification is the way to go, and any platform of any
> interest supports it these days. Having said that we'll still need a

> Maybe have a normal full scan and a checksum full scan and an option
> that says every N scans is a checksum scan. This should encourage

I don't think that's the way to go. MD5s not only provide enhanced change detection, but also remote store content verification on every backup cycle. Depending on media rotation strategy, bbstored soft limit settings, etc. one might have to treat the very last backup and the only usable backup. At any rate, there are situations when either a backup is 100% verified, or counts for no backup at all...

Gary






       
____________________________________________________________________________________
Don't get soaked.  Take a quick peak at the forecast
with the Yahoo! Search weather shortcut.
http://tools.search.yahoo.com/shortcuts/#loc_weather