[Box Backup] Backup File Order

[Johann Glaser]

Hi!

> Sorry, not at the moment. Maybe we could add something like a 
> BackupFilesFirst directive under a location which causes the list of files 
> in each directory to be scanned twice, and on the first pass only files 
> matching BackupFilesFirst are backed up.

I would like to generalize this. Probably somebody needs some files
backed up last, or even both. So, I propose to offer directives to
specify files/directories in a user-defined order which are then backed
up in the that order.

The first issue is what to do with files which are not captured by any
of the regexps? 

The second issue is what to do with files which are captured multiple
times? They should be backed up only once. The "best" way would be to
use the least-specific directive, e.g. in the example below "./current"
and "./lastfile" both are captured by "[.]/.*" too, but should be
excluded from this.

For easier configuration I propose the following additional directive
which specifies the order within one subdirectory. 
  BackupFilesOrder = (reverse) (any|sorted|alphabetical|numeric|timestamp|size)
 - "any" uses the files as they come from the filesystem
 - "sorted" should be a normal ASCII-code sort
 - "alphabetical" should consider the locale to sort
 - "numeric" sorts numbers by their value, e.g. "2" is before "10"
 - "timestamp" sorts the files by their mtime
 - "size" sorts the files by their size
 - "reverse" reverses the order

To specify whether files or directories should be mixed or separated, I
propose the following directive:
  BackupSubdirs = (mixed|first|last)
 - "mixed" is to consider them in the order coming from the filesystem
 - "first" backups subdirectories before the files in the directories
 - "last" backups the subdirs after all files in the directory

It seems hat BoxBackup currently doesn't support Pre- and Post-Scripts.
I propose the following directives
  PreScript = /backup/scripts/bb-mysql-full-pre
  PostScript = /backup/scripts/bb-mysql-full-post
The executed scripts should get the directory they are applied to as a
parameter.

It would be even better if that directives could be applied to specific
subdirectories and groups of files, e.g.

BackupLocations {
  data {
    Path = /data
    ExcludeDir = /data/mysql
    Subdirectory {
      PathRegex = /data/svn/repositories/[a-zA-Z0-9]+/db/
      BackupSubdirs = last
      BackupFilesOrder = sorted
      PreScript = /backup/scripts/bb-svn-full-pre
      Subdirectory {
        Path = ./current
        Priority = 100
      }
      Subdirectory {
        PathRegex = [.]/.*
        Priority = 300
        Subdirectory {
          PathRegex = [.]/(revs|revprops)/.*
          BackupFilesOrder = reverse numeric
        }
      }
      Subdirectory {
        Path = ./lastfile
        Priority = 500
      }
    }
  }
}

I don't like the "Subdirectory" directive because it should also be
applicable to files or a list of files, but I couldn't think of another
one.

I'll put this to the feature request Wiki.

> > If it is currently not possible, what is the current order of backups? 
> > Subdirs-first or files-first? Are the files sorted alphabetically before 
> > backed up or are they taken as they come from the filesystem?
> 
> As they come from the filesystem. We never had a need to sort them before.

I see. Does that mean that directories are also mixed with the files?

Bye
  Hansi

-- 
Johann Glaser                          <glaser@ict.tuwien.ac.at>
             Institute of Computer Technology, E384
Vienna University of Technology, Gusshausstr. 27-29, A-1040 Wien
Phone: ++43/1/58801-38444                Fax: ++43/1/58801-38499