[Box Backup] Re: boxbackup digest, Vol 1 #57 - 2 msgs

[boxbackup@fluffy.co.uk]

> > However the Oracle top level item that I created incorrectly is still
> > there. Should it go away on it's own as well, it's completely gone from
> > my configuration file now?
>
> Now this, I must confess, is something I've yet to do. I need to remove
> removed locations from the store. I think I've been delaying because I
> feel I should wait a day or so to make sure that the adminstrator
> really wanted to do it, and that's a more complex thing to do.

Why don't you make it go a away when things hit the soft limit as with the
other stuff? That way the admin has a while to discover their mistake... I
suppose something in the client log the first time or two might be good
as well. I run logcheck on my Debian machine, and so I see the various
messages that BoxBackup sends out...

>
> >
> > Do you think people need a way to purposely delete backups? Say they
> > backed up something they really, really, don't want a copy of? You
> > seem to
> > have thought about this carefully what's your opinion? (Something like
> > the
> > Sarbanes-Oxely stuff in the USA).
> >
> > I can see that something like that could really cause problems, but may
> > also be useful if people are managing their own backups (for instance
> > email retention laws where you probably want it to disappear on the
> > crack
> > of midnight the first day you can delete it). I also get the impression
> > you need to be able to hold certain items back from deletion sort of
> > indefinitely say if the court has ordered you to keep it for some
> > reason.
>
> I have been more interesting in preserving data than destroying it, so
> controlled deletion didn't really occur to me. But you're right, it is
> useful.
>
> There does need to be a way of retaining data more carefully. When I
> add the "mark" feature, you'll be able to specify how many marked files
> should exist, so you could mark weekly, and say you want to keep at
> least 6 weeks of marked files.

The Sarbanes-Oxely stuff though would require you to absolutely guarantee
that something could not get deleted until it was untagged.

>
> >
> > Do you think a config item for max retention time might be useful?
>
> Sounds a sensible start.

And again for Sarbanes-Oxely it would have to get nuked immediately once
it expired, it could not wait until you hit the soft limit.

I suppose you have the wrinkle of what to do with the file if it still
exists on the client, not backing it up could be counter-intuitive, but I
guess you could only keep one copy or something in that case. Actually I
think that's the wrong way of looking at it, see below:

Maybe if the location had a min-retention-time and a max-retention-time,
that way you would know the file (older copies of the current file on
client disk as well) would be kept for at least the minimum time period,
then all the older copies would for sure get nuked at the
max-retention-time, the current backup of course would be retained for
min-retention-time. Kind of like a conveyor belt with stuff falling off
the end.

The min-retention-time would supersede any garbage collection, you would
know for sure that file would exist for that time period, if you run out
of space, then you deal with it...

Again the max-retention-time would nuke the backups immediately after they
expired, it wouldn't wait for garbage collection.

It might be interesting to have an API where you could programatically
flag files to be unremovable, but then, maybe it makes more sense to have
a regexp in the config file where you could specify files or directories
as being undeletable. If someone needed to modify things, they could
modify the config file from whatever program was managing the backup
stuff.

Finally, to comply with something like Sarbanes-Oxely, that might not be
good enough, you may need the ability to mark a particular backup(s) of a
file as being undeletable, but ensure the rest follow the current rules.
After all, you want to nuke the stuff that is covered by that as quickly
as possible so it can't be requested by your opponent. However that would
probably complicate things, so going with the pure conveyor belt method
would probably be better as a start.

Finally, we would assume that the client has something on the client
machine that would handle deleting files that had expired, so if the file
needed to disappear, they would handle that. At that point the file's
backups would follow the retention rules. Basically limit our scope to
what we can reasonably control and worry about.

The only other thing that comes to mind is perhaps some sort of optional
logging when expired files and so on are deleted, that way someone/(some
program) could be sure that the retention rules were being followed.

> > Is there some way to list how much space you are allowed to have on the
> > drive, along with how much you are using from the backup query client?
>
> I will add a command to bbackupquery -- I always use bbstoreaccounts on
> the server to find out such things, but this is not possible if you
> don't have that access to the server.

I don't think you have really said this, but I see great potential in this
software being used as a *utility* where someone rents out a server as a
bit-bucket to others for a monthly fee, having proper retention rule hooks
would allow it to be used as such in more situations.

I don't personally have any real intention of using the software that way,
but I can see the retention rules being useful to my friends who have
businesses that I might back up onto my server.


Rick