[Box Backup] Files are uploaded/patched daily even though they have not changed in weeks.

[Chris Wilson]

Hi Matt,

On Tue, 8 Apr 2008, Matt Brown wrote:

> I am seeing a rather strange issue, which I am sure is down to what I 
> have done :-)

I doubt it, thanks for reporting it, I will definitely investigate.

> So I created a new bbstored on the remote server (a 1 & 1 root server) 
> and copied the same raw keys etc to it and the backup ran fine, however 
> due to 1 & 1 randomly reseting connections to their servers (presumably 
> to avoid the root servers being used in such a way)

Damn, that sucks. I had no idea 1&1 would do that. I haven't seen any 
evidence of it happening with a root server that I use there. What does it 
look like? Is it long-running or idle connections that get terminated, or 
any connection? Could it be a firewall state table timeout?

> I was forced to point the data back to the original server so at least 
> the data was being backed up - (the pipe would fail and box would fail 
> to continue once the connection failed without restarting the sync, 
> leaving me with incomplete backups).

I would definitely like to know more about this. Can you reproduce this 
issue? What version of Box were you using? What do you get in the logs 
after the aborted connection?

> This is where the strange issue is, every day I get an email with the 
> contents of the backup via the boxreporter.py script, and the contents 
> are the same + changed files for that day. To make sure it was not a 
> reporting issue, I ran the sync by hand and monitored the logs. Sure 
> enough the same files are upload or patched - but the files in question 
> have not changed for over a week now.
> 
> It appears somehow, the files in question are always seen as changed and 
> forces a sync or is cached (well thats the behavior) ?
> 
> Any ideas how I can debug this to work out what is going on ?

Could you try running bbackupd with LogAllFileAccess = yes and the -V 
option and see what it says about these files?

Could it be that their modification date is in the future, or is being 
updated?

> Could it be down to the maxdiffing time ?

It could be, although I think that if Box thinks the file has changed then 
it will calculate and upload a patch, even if it's an empty patch (i.e. 
there were no actual changes), so it probably happened before that, when 
Box decided that it thought the file had been changed.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\ _/_/_/_//_/___/ | We are GNU : free your mind & your software |