[Box Backup] I have the fear....

Tue, 13 Mar 2007 12:39:46 -0400

--YZ5djTAD1cGYuMQK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Thanks for the quick, helpful reponse.

On Tue, Mar 13, 2007 at 09:44:30AM +0000, Chris Wilson wrote:
> On Mon, 12 Mar 2007, Ben Bennett wrote:
> 
> >But when I checked into /etc/genkernel.conf in detail I saw that there 
> >were two versions in the backup repository, the newer one had a date 
> >shortly after the source file modify time (so it spotted the change), 
> >but the contents of both versions were identical, so something got 
> >messed up.
> 
> How recently was it modified? 

It was modified a few weeks ago (on Feb 19th).  Ah... but here is a
possibility.  This system was updated, and it is running gentoo which
doesn't change any config files if they have been modified... it
places a ._cfg... file in the dir in question and then you update it
with a tool.  If you accept hte new version without changes, it is
just moved into place.  Could that have confused something?

> Possibly Box Backup isn't running any more, or is configured not to
> back up files for a long time after they change?

BoxBackup is running (but may have been stopped for a week).  I am
backing up to an external USB device which one of my kids had
"helpfully" turned off.  Is the boxbackup state stored somewhere other
than the boxbackup dir?  Could something have been confused and
thought it had been written, yet the blocks were unable to be stored?

> Please post your bbackupd.conf file.

Attached.

> >Are either the bad messages, or the messed up backups known problems?
> 
> The bad messages are, but broken backups most definitely are not.

Are the subversion versions safe to run?  Is there an expected release
anytime soon?  Do I need to submit a patch for the bad messages?

> >Is there a fix?  Is there any additional information that I can provide 
> >that would help debug this?
> 
> I'd be happy to try to help you fix this. First let's see your config, 
> stat /etc/genkernel.conf and list both versions of /etc/genkernel.conf on 
> the server using bbackupquery.

Ah, unfortunately last night I touched the files to try to get them to
back up :-(

I can give you part of what you need (I switched to host.conf because
it is a little smaller):

# bbackupquery 'list -dots /etc' quit | grep host.conf
00000081 f--o-- 2006-11-16T04:52:58 00001 host.conf
0000171a f-X--- 2007-02-19T21:53:11 00001 ._cfg0000_host.conf
0000171d f--o-- 2007-02-19T21:53:11 00001 host.conf
00006cbc f----- 2007-03-12T23:54:01 00001 host.conf

# ls -l /etc/host.conf
-rw-r--r-- 1 root root 936 Mar 12 19:54 /etc/host.conf

Then I restored all of the above files with:
# bbackupquery 'cd /etc' 'get -i 00006cbc 00006cbc' quit

And copied in the current host.conf

And then (I added the comments following #):
# sum *
01107     1 00000081    # Original host.conf at 1st backup
55758     1 0000171a    # The ._cfg... file
01107     1 0000171d    # The "new" config
55758     1 00006cbc    # Post-touch from yesterday
55758     1 host.conf   # The file on disk

The first column of the sum output shows the checksum of the file, so
when the numbers are the same, the contents are the same.

You can see from the above the following history:
1) host.conf (id 00000081/sum 01107) existed when the first backup was
   run.  This was the state before I updated the system and host.conf
   needed to be changed
2) ._cfg0000_host.conf (0000171a/55758) was created pending the run
   of the tool that updates the real config files.  This has the new
   contents of the file.
3) Update tool ran, moving ._cfg0000_host.conf to host.conf, and
   boxbackup spotted the change, but preserved the old contents as 
   (0000171d/1107), exactly the same contents as the predecessor version.
4) I touched host.conf
5) boxbackup correctly updated the server (00006cbc/55758)

	     Thanks,

		-ben
--YZ5djTAD1cGYuMQK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="bbackupd.conf"

StoreHostname = ayup.limey.net
AccountNumber = 0x1
KeysFile = /etc/boxbackup/bbackupd/1-FileEncKeys.raw

CertificateFile = /etc/boxbackup/bbackupd/1-cert.pem
PrivateKeyFile = /etc/boxbackup/bbackupd/1-key.pem
TrustedCAsFile = /etc/boxbackup/bbackupd/serverCA.pem

DataDirectory = /var/bbackupd

# This script is run whenever bbackupd encounters a problem which requires
# the system administrator to assist:
# 1) The store is full, and no more data can be uploaded.
# 2) Some files or directories were not readable.
# The default script emails the system administrator.

NotifyScript = /etc/boxbackup/bbackupd/NotifySysadmin.sh

# A scan of the local discs will be made once an hour (approximately).
# To avoid cycles of load on the server, this time is randomly adjusted by a small
# percentage as the daemon runs.

UpdateStoreInterval = 3600

# A file must have been modified at least 6 hours ago before it will be uploaded.

MinimumFileAge = 21600

# If a file is modified repeated, it won't be uploaded immediately in case it's modified again.
# However, it should be uploaded eventually. This is how long we should wait after first noticing
# a change. (1 day)

MaxUploadWait = 86400

# Files above this size (in bytes) are tracked, and if they are renamed they will simply be
# renamed on the server, rather than being uploaded again. (64k - 1)

FileTrackingSizeThreshold = 65535

# The daemon does "changes only" uploads for files above this size (in bytes).
# Files less than it are uploaded whole without this extra processing.

DiffingUploadSizeThreshold = 8192

# The limit on how much time is spent diffing files. Most files shouldn't take very long,
# but if you have really big files you can use this to limit the time spent diffing them.
# * Reduce if you are having problems with processor usage.
# * Increase if you have large files, and think the upload of changes is too large and want
#   to spend more time searching for unchanged blocks.

MaximumDiffingTime = 20

# Uncomment this line to see exactly what the daemon is going when it's connected to the server.

# ExtendedLogging = yes

# Use this to temporarily stop bbackupd from syncronising or connecting to the store.
# This specifies a program or script script which is run just before each sync, and ideally
# the full path to the interpreter. It will be run as the same user bbackupd is running as,
# usually root.
# The script prints either "now" or a number to STDOUT (and a terminating newline, no quotes).
# If the result was "now", then the sync will happen. If it's a number, then the script will
# be asked again in that number of seconds.
# For example, you could use this on a laptop to only backup when on a specific network.

# SyncAllowScript = /path/to/intepreter/or/exe script-name parameters etc

# Where the command socket is created in the filesystem.

CommandSocket = /var/run/bbackupd.sock

# Uncomment the StoreObjectInfoFile to enable the experimental archiving
# of the daemon's state (including client store marker and configuration)
# between backup runs. This saves time and increases efficiency when
# bbackupd is frequently stopped and started, since it removes the need
# to rescan all directories on the remote server. However, it is new and
# not yet heavily tested, so use with caution.

# StoreObjectInfoFile = /var/bbackupd/bbackupd.state

Server
{
	PidFile = /var/run/bbackupd.pid
}

# 
# BackupLocations specifies which locations on disc should be backed up. Each
# directory is in the format
# 
# 	name
# 	{
# 		Path = /path/of/directory
# 		(optional exclude directives)
# 	}
# 
# 'name' is derived from the Path by the config script, but should merely be
# unique.
# 
# The exclude directives are of the form
# 
# 	[Exclude|AlwaysInclude][File|Dir][|sRegex] = regex or full pathname
# 
# (The regex suffix is shown as 'sRegex' to make File or Dir plural)
#
# For example:
# 
# 	ExcludeDir = /home/guest-user
# 	ExcludeFilesRegex = *.(mp3|MP3)$
# 	AlwaysIncludeFile = /home/username/veryimportant.mp3
# 
# This excludes the directory /home/guest-user from the backup along with all mp3
# files, except one MP3 file in particular.
# 
# In general, Exclude excludes a file or directory, unless the directory is
# explicitly mentioned in a AlwaysInclude directive.
# 
# If a directive ends in Regex, then it is a regular expression rather than a
# explicit full pathname. See
# 
# 	man 7 re_format
# 
# for the regex syntax on your platform.
# 

BackupLocations
{
	etc
	{
		Path = /etc
		ExcludeFile = /etc/boxbackup/bbackupd/1-FileEncKeys.raw
	}
	pictures
	{
		Path = /home/fiji/public_html/pictures
  	}
}

--YZ5djTAD1cGYuMQK--