[Box Backup] Unnecessary upload and logging compares

[Chris Wilson]

Hi Roy,

On Fri, 18 Sep 2009, Roy wrote:

> I think my messages are lost in between the activity this month.

Sorry about that, and thanks for reminding me.

> But I still have two questions/problems.
>
> One Box Backup installation on Windows XP, keeps uploading a lot of data. But 
> not much has changed. Always the first scan of files, results in the biggest 
> upload of the day. So I think something is wrong with the first scan...
>
> I see in the logfile the following:
> MaximumDiffingTime reached - suspending file diff

This probably means that a large file was being diffed, that it took too 
long (more than MaximumDiffingTime seconds) and that the entire file was 
uploaded instead. If the file really is large, you will probably reduce 
bandwidth usage by increasing MaximumDiffingTime.

> Not deleting unused entries - none in list
> Not deleting unused entries - too early (2899 seconds remaining) -> because I 
> removed a location

These should not mean that any additional bandwidth was used.

> How can I see, what is causing this big uploads? Which files are uploaded?

Try enabling the "LogAllFileAccess = yes" option on the client, it should 
tell you which files were uploaded, and whether they were completely 
uploaded or as patches. E.g. you should see messages like:

 	Uploading complete file: /foo/bar/baz
 	Uploading patch to file: /foo/bar/whee

I'm in the process of adding code to trunk that will tell you exactly how 
many bytes were uploaded for each file when using LogAllFileAccess.

> Upload stats:
> Today: 147 MB
> Yesterday: 1503 MB (this is when I added a lot of photos, so this is normal)
> 16-9: 331 MB
> 15-9: 58 MB
> 14-9: 147 MB

How were these calculated, by the way?

> Next problem is that I want to log the output of a compare (CLI), but I 
> cannot seem to do it. Because "> output.txt" doesn't work. Can I put 
> something in the config file to log this? Do I need to enable: 
> ExtendedLogging? And where is the data logged then?

ExtendedLogging will probably not help you much unless you're very 
familiar with the Box Backup protocol. However you should be able to use 
the "-o" option to bbackupquery to specify that logs should be written to 
a file.

I've added notes to the FAQ for both problems, starting here:

http://www.boxbackup.org/trac/wiki/FAQ#how-can-i-find-out-why-box-backup-is-uploading-so-much-data

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <chris+sig at qwirx.com> Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\__/_/_/_//_/___/ | We are GNU : free your mind & your software |