[Box Backup] Issue with Incremental backup of mail folder

[Imran Niazi]

Hello Chris, 

I really appreciate taking the time to answer my email in detail.

> Did you try listing with the -o option? (show old versions of files).

Yeah, I usually use 'ls -dots'  

> Truncation or replacement of the file should not have any effect on 
> the way that backups work, i.e. the number of old versions which are 
> kept. The only things that will affect that are the size of the 
> increments (daily changes) and the available space on the store for 
> this account (versus the amount of data in current files, i.e. how 
> much space is available for storing old versions of files).

Yeah, actually I forgot to note the current store info:

                  Account ID: 00040404
              Last object ID: 660916
                 Blocks used: 9157304 (35770.72Mb)
    Blocks used by old files: 29429 (114.96Mb)
Blocks used by deleted files: 5389922 (21054.38Mb)
  Blocks used by directories: 25831 (100.90Mb)
            Block soft limit: 9175040 (35840.00Mb)
            Block hard limit: 12058624 (47104.00Mb)
         Client store marker: 1176226959000000

I guess boxbackup or housekeeping puts more emphasis on deleted files and less
on old files.  This is a mail server, and files are the mbox format files
(i.e. one big mail file) for each user.  the user in question, her mailbox was
probably about 40 megs.  And she receives about 10 or 15 emails a day.  so the
file is constantly changing, and only being sent to bbstored every 24 hours
(which is what the threshold is set to now).  

Oh, btw, the bbstored process in this case has been running since late 2006,
but most probably since october 2006, thats when I upgraded (or reinstalled
after upgrading the OS, I forget which).  Btw I've been running boxbackup, for
three years, I think when 0.5 was released.

On some of the other clients, the store looks little different:

                  Account ID: 00010101
              Last object ID: 5867185
                 Blocks used: 34586439 (135103.28Mb)
    Blocks used by old files: 845160 (3301.41Mb)
Blocks used by deleted files: 21068380 (82298.36Mb)
  Blocks used by directories: 166764 (651.42Mb)
            Block soft limit: 34586440 (135103.28Mb)
            Block hard limit: 38046108 (148617.61Mb)
         Client store marker: 1171203234000000

                  Account ID: 00020202
              Last object ID: 1380899
                 Blocks used: 26214040 (102398.59Mb)
    Blocks used by old files: 5824767 (22753.00Mb)
Blocks used by deleted files: 17815456 (69591.62Mb)
  Blocks used by directories: 78734 (307.55Mb)
            Block soft limit: 26214400 (102400.00Mb)
            Block hard limit: 31457280 (122880.00Mb)
         Client store marker: 1175843302000000

                  Account ID: 00030303
              Last object ID: 5654517
                 Blocks used: 40366239 (157680.62Mb)
    Blocks used by old files: 1712 (6.69Mb)
Blocks used by deleted files: 9633809 (37632.07Mb)
  Blocks used by directories: 148124 (578.61Mb)
            Block soft limit: 40366240 (157680.62Mb)
            Block hard limit: 44403888 (173452.69Mb)
         Client store marker: 1177484669000000

                  Account ID: 00050505
              Last object ID: 88868
                 Blocks used: 638507 (2494.17Mb)
    Blocks used by old files: 258480 (1009.69Mb)
Blocks used by deleted files: 13590 (53.09Mb)
  Blocks used by directories: 17988 (70.27Mb)
            Block soft limit: 15728640 (61440.00Mb)
            Block hard limit: 18350080 (71680.00Mb)
         Client store marker: 1164184762000000


> Yes, that would be a cool feature. Boxi has something like that, 
> where you can restore a file "as of" a specific time, and you will 
> get last version backed up before that time.

Yeah?  But in this case, the file was backed up every 24 hours, but only
version I can get is the latest one (24 hours ago).  (just to be clear)
 
> rdiff-backup does something like that already, but it's not 
> encrypted. For what it's worth, I currently use rdiff-backup for all 
> of my backups (but never to untrusted machines).

I haven't used rdiff-backup but have heard of it.  I've been using boxbackup
with great success in the past 3 years.  I've had to restore couple of
harddrives that crashed, and I was able to get data pretty quickly.  I mainly
like the feature of boxbackup that the items in question are online and I can
browse the directory structure etc.  I was used to the commercial solutions
which provided the nice directory listing etc. but some of the open source
backup programs I saw, were snapshot based, and I would have to restore the
backup to a dir to see what files that were there etc.  Plus I like the fact
that the client distributes the load of backups during the day (assuming that
files are changing a few at a time etc.)  Not to mention that content is
encyrpted etc.

Thanks again,

Imran Niazi
http://www.niazi.net