[Box Backup] Unnecessary upload and logging compares

[Roy]

Chris Wilson wrote:
> Hi Roy,
>
> On Fri, 18 Sep 2009, Roy wrote:
>
>> I think my messages are lost in between the activity this month.
>
> Sorry about that, and thanks for reminding me.
>
>> But I still have two questions/problems.
>>
>> One Box Backup installation on Windows XP, keeps uploading a lot of 
>> data. But not much has changed. Always the first scan of files, 
>> results in the biggest upload of the day. So I think something is 
>> wrong with the first scan...
>>
>> I see in the logfile the following:
>> MaximumDiffingTime reached - suspending file diff
>
> This probably means that a large file was being diffed, that it took 
> too long (more than MaximumDiffingTime seconds) and that the entire 
> file was uploaded instead. If the file really is large, you will 
> probably reduce bandwidth usage by increasing MaximumDiffingTime.
Don't know which file this was, so I'm not sure if I should make 
MaximumDiffingTime higher. But that may be a solution.
BTW these log entries are to demonstrate that there aren't any weird 
messages that point to excessive uploads.
>
>> Not deleting unused entries - none in list
>> Not deleting unused entries - too early (2899 seconds remaining) -> 
>> because I removed a location
>
> These should not mean that any additional bandwidth was used.
>
>> How can I see, what is causing this big uploads? Which files are 
>> uploaded?
>
> Try enabling the "LogAllFileAccess = yes" option on the client, it 
> should tell you which files were uploaded, and whether they were 
> completely uploaded or as patches. E.g. you should see messages like:
>
>     Uploading complete file: /foo/bar/baz
>     Uploading patch to file: /foo/bar/whee
>
Will do this. But where is it logged then? In the eventviewer? I hope 
not, because you cannot easily read that, like you can with a logfile.
> I'm in the process of adding code to trunk that will tell you exactly 
> how many bytes were uploaded for each file when using LogAllFileAccess.
>
>> Upload stats:
>> Today: 147 MB
>> Yesterday: 1503 MB (this is when I added a lot of photos, so this is 
>> normal)
>> 16-9: 331 MB
>> 15-9: 58 MB
>> 14-9: 147 MB
>
> How were these calculated, by the way?
On the server I sum those numbers on a daily basis. And in the 
eventviewer I see the numbers of the individuals backup runs, so I know 
that the first one is the biggest.
>
>> Next problem is that I want to log the output of a compare (CLI), but 
>> I cannot seem to do it. Because "> output.txt" doesn't work. Can I 
>> put something in the config file to log this? Do I need to enable: 
>> ExtendedLogging? And where is the data logged then?
>
> ExtendedLogging will probably not help you much unless you're very 
> familiar with the Box Backup protocol. However you should be able to 
> use the "-o" option to bbackupquery to specify that logs should be 
> written to a file.
>
> I've added notes to the FAQ for both problems, starting here:
>
> http://www.boxbackup.org/trac/wiki/FAQ#how-can-i-find-out-why-box-backup-is-uploading-so-much-data 
>
>
> Cheers, Chris.
I didn't know the -o option. Thanks for that!

Cheers,

Roy