[Box Backup] Backup File Order

[Chris Wilson]

Hi Johann,

On Thu, 13 Sep 2007, Johann Glaser wrote:

> I would like to generalize this. Probably somebody needs some files 
> backed up last, or even both. So, I propose to offer directives to 
> specify files/directories in a user-defined order which are then backed 
> up in the that order.
>
> The first issue is what to do with files which are not captured by any 
> of the regexps?
>
> The second issue is what to do with files which are captured multiple 
> times? They should be backed up only once. The "best" way would be to 
> use the least-specific directive, e.g. in the example below "./current" 
> and "./lastfile" both are captured by "[.]/.*" too, but should be 
> excluded from this.
>
> For easier configuration I propose the following additional directive
> which specifies the order within one subdirectory.
>  BackupFilesOrder = (reverse) (any|sorted|alphabetical|numeric|timestamp|size)
> - "any" uses the files as they come from the filesystem
> - "sorted" should be a normal ASCII-code sort
> - "alphabetical" should consider the locale to sort
> - "numeric" sorts numbers by their value, e.g. "2" is before "10"
> - "timestamp" sorts the files by their mtime
> - "size" sorts the files by their size
> - "reverse" reverses the order
>
> To specify whether files or directories should be mixed or separated, I
> propose the following directive:
>  BackupSubdirs = (mixed|first|last)
> - "mixed" is to consider them in the order coming from the filesystem
> - "first" backups subdirectories before the files in the directories
> - "last" backups the subdirs after all files in the directory
>
> It seems hat BoxBackup currently doesn't support Pre- and Post-Scripts.
> I propose the following directives
>  PreScript = /backup/scripts/bb-mysql-full-pre
>  PostScript = /backup/scripts/bb-mysql-full-post
> The executed scripts should get the directory they are applied to as a
> parameter.
>
> It would be even better if that directives could be applied to specific
> subdirectories and groups of files, e.g.
>
> BackupLocations {
>  data {
>    Path = /data
>    ExcludeDir = /data/mysql
>    Subdirectory {
>      PathRegex = /data/svn/repositories/[a-zA-Z0-9]+/db/
>      BackupSubdirs = last
>      BackupFilesOrder = sorted
>      PreScript = /backup/scripts/bb-svn-full-pre
>      Subdirectory {
>        Path = ./current
>        Priority = 100
>      }
>      Subdirectory {
>        PathRegex = [.]/.*
>        Priority = 300
>        Subdirectory {
>          PathRegex = [.]/(revs|revprops)/.*
>          BackupFilesOrder = reverse numeric
>        }
>      }
>      Subdirectory {
>        Path = ./lastfile
>        Priority = 500
>      }
>    }
>  }
> }
>
> I don't like the "Subdirectory" directive because it should also be
> applicable to files or a list of files, but I couldn't think of another
> one.
>
> I'll put this to the feature request Wiki.

Both proposals sound interesting, but I can't help feeling that this is 
perhaps a little over-complicated. Do you have a use case that actually 
requires all this?

For example, I think you said that provided we back up the files in the 
correct order, we can back up a live SVN filesystem and we don't need to 
make a hot copy? If so, then why the need for the pre and post scripts? 
What would they do?

What does the Priority directive do? Override the default sort order? 
Should there be a sort-by-priority BackupFilesOrder?

>> As they come from the filesystem. We never had a need to sort them 
>> before.
>
> I see. Does that mean that directories are also mixed with the files?

No, we split the list into files and directories and do deep recursion 
first (I think).

Cheers, Chris.
-- 
_____ __     _
\  __/ / ,__(_)_  | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |