[Box Backup] Files are uploaded/patched daily even though they have not changed in weeks.

[Chris Wilson]

Hi Matt,

On Tue, 8 Apr 2008, Matt Brown wrote:

> > >due to 1 & 1 randomly reseting connections to their servers 
> > >(presumably to avoid the root servers being used in such a way)
> >
> >Damn, that sucks. I had no idea 1&1 would do that. I haven't seen any 
> >evidence of it happening with a root server that I use there. What does 
> >it look like? Is it long-running or idle connections that get 
> >terminated, or any connection? Could it be a firewall state table 
> >timeout?
> 
> Seems to be idle timeout, if I leave an ssh session open it gets closed. 
> I do use their firewall option, so it could be a state thing.

Yeah, I don't use their firewall, and I've had commentions open for 24 
hours. I'll open one now and see how long it lasts.

> > >I was forced to point the data back to the original server so at 
> > >least the data was being backed up - (the pipe would fail and box 
> > >would fail to continue once the connection failed without restarting 
> > >the sync, leaving me with incomplete backups).
> >
> >I would definitely like to know more about this. Can you reproduce this
> >issue? What version of Box were you using? What do you get in the logs
> >after the aborted connection?
> 
> Well currently the version I am using is  v0.11rc2 but I noticed I am using
> the state db
> 
> i.e StoreObjectInfoFile = /var/bbackupd/bbackupd.state
> 
> It does say ...
> 
> #This saves time and increases efficiency when
> # bbackupd is frequently stopped and started, since it removes the need
> # to rescan all directories on the remote server. However, it is new and
> # not yet heavily tested, so use with caution.
> 
> So this could be the culprit .. tho it has behaved until now ....

I'm afraid this option is still not well tested, but let's see what the 
logs say.

> >Could you try running bbackupd with LogAllFileAccess = yes and the -V 
> >option and see what it says about these files?
> >
> Yep got all these enabled, as used for the reporting function ....
> 
> >Could it be that their modification date is in the future, or is being
> >updated?
> >
> Timestamps look fine.

i.e. not changing?

Please could you send me the logs for these files off-list?

> > >Could it be down to the maxdiffing time ?
> >
> >It could be, although I think that if Box thinks the file has changed then
> >it will calculate and upload a patch, even if it's an empty patch (i.e.
> >there were no actual changes), so it probably happened before that, when
> >Box decided that it thought the file had been changed.
> 
> I could always blat the store for this account - which would clear the 
> issue, however I am curious as to why it thinks the files are not sync'd

Me too, and worried, please don't blat the account as we may need it for 
evidence and to test any potential fixes.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Ruby/Perl/SQL Developer |
\ _/_/_/_//_/___/ | We are GNU : free your mind & your software |