[Box Backup] I have the fear....

[Chris Wilson]

Hi Ben,

On Tue, 13 Mar 2007, Ben Bennett wrote:

> Thanks for the quick, helpful reponse.

You're welcome, I hope that I can help.

>> How recently was it modified?
>
> It was modified a few weeks ago (on Feb 19th).  Ah... but here is a
> possibility.  This system was updated, and it is running gentoo which
> doesn't change any config files if they have been modified... it
> places a ._cfg... file in the dir in question and then you update it
> with a tool.  If you accept hte new version without changes, it is
> just moved into place.  Could that have confused something?

It should not have confused anything. If the new version is renamed over 
the old one, then Box Backup should detect this, and mark this as deleting 
the old version, and renaming the ._cfg file over the old one. So you 
should not see the ._cfg file in the directory listing, just a new 
(current) version of genkernel.conf, and the old (deleted) version.

> BoxBackup is running (but may have been stopped for a week).

If it was stopped, then it would not be backing up. Once started again, it 
should catch up with any changes that it has missed.

> I am backing up to an external USB device which one of my kids had 
> "helpfully" turned off.  Is the boxbackup state stored somewhere other 
> than the boxbackup dir?

Usually the full state is kept only in memory, but the list of object IDs 
and inodes (the ID map) is written to a BDB database in the boxbackup data 
directory if Box Backup has been compiled with BDB support.

There is a new StoreObjectInfoFile directive in 0.10 which backs up all 
state information to the specified file, which could be located anywhere.

The cost of "loss of state" due to a Box Backup stopping or restarting is 
usually just some extra bandwidth use on the first pass after restart, as 
Box Backup reads the info on all remote directories from the store to 
refresh its local cache.

> Could something have been confused and thought it had been written, yet 
> the blocks were unable to be stored?

It is slightly possible, but I have not seen or heard about any such 
behaviour.

Normally, the ID maps should be committed atomically and only after a 
successful sync, which should mean that they cannot get out of sync. If 
any error occurs, they should be deleted and recached from the store on 
the next backup run.

> Are the subversion versions safe to run?

I think so, but I don't use Box Backup in production yet, myself. Several 
users use my general tree (for win32) without problems, and the merge tree 
is almost the same as the general tree, but it's believed to work on Unix.

All tests currently pass on the merge tree, which makes me think that it's 
as safe as any release version, although of course it is not heavily 
tested.

I would strongly recommend that you do exactly what you already are doing, 
i.e. regularly verify your backups, whichever version of Box Backup (or 
any backup software) you run. If your backups verify OK, and you have a 
secure offline copy of your encryption keys, then I'd say that you are 
safe.

> Is there an expected release anytime soon?

We do not have a release scheduled as far as I know. I would very much 
like to see my general (win32) tree cleaned up, merged and released as 
0.11 soon, but that depends on others helping me by reviewing my new code, 
so I can't make any promises about this.

> Do I need to submit a patch for the bad messages?

I don't think so, I think I have already fixed it in my tree. Feel free to 
give it a try and let me know if the problem still happens.

> Ah, unfortunately last night I touched the files to try to get them to
> back up :-(

Did it work?

> I can give you part of what you need (I switched to host.conf because
> it is a little smaller):
>
> # bbackupquery 'list -dots /etc' quit | grep host.conf
> 00000081 f--o-- 2006-11-16T04:52:58 00001 host.conf
> 0000171a f-X--- 2007-02-19T21:53:11 00001 ._cfg0000_host.conf
> 0000171d f--o-- 2007-02-19T21:53:11 00001 host.conf
> 00006cbc f----- 2007-03-12T23:54:01 00001 host.conf
>
> # ls -l /etc/host.conf
> -rw-r--r-- 1 root root 936 Mar 12 19:54 /etc/host.conf
>
> Then I restored all of the above files with:
> # bbackupquery 'cd /etc' 'get -i 00006cbc 00006cbc' quit
>
> And copied in the current host.conf
>
> And then (I added the comments following #):
> # sum *
> 01107     1 00000081    # Original host.conf at 1st backup
> 55758     1 0000171a    # The ._cfg... file
> 01107     1 0000171d    # The "new" config
> 55758     1 00006cbc    # Post-touch from yesterday
> 55758     1 host.conf   # The file on disk
>
> The first column of the sum output shows the checksum of the file, so
> when the numbers are the same, the contents are the same.
>
> You can see from the above the following history:
> 1) host.conf (id 00000081/sum 01107) existed when the first backup was
>   run.  This was the state before I updated the system and host.conf
>   needed to be changed
> 2) ._cfg0000_host.conf (0000171a/55758) was created pending the run
>   of the tool that updates the real config files.  This has the new
>   contents of the file.
> 3) Update tool ran, moving ._cfg0000_host.conf to host.conf, and
>   boxbackup spotted the change, but preserved the old contents as
>   (0000171d/1107), exactly the same contents as the predecessor version.
> 4) I touched host.conf
> 5) boxbackup correctly updated the server (00006cbc/55758)

OK, so it appears that Box Backup did not notice that ._cfg0000_host.conf 
was moved over host.conf, and therefore did not back up host.conf again, 
until you touched the file. Is that your understanding as well?

If so, then it does indeed look like a bug in Box Backup. Thanks for 
reporting it, I will investigate.

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |