[Box Backup] Re: boxbackup digest, Vol 1 #57 - 2 msgs

[Ben Summers]

On 19 Apr 2004, at 19:47, rprice@freeshell.org wrote:

>>> However the Oracle top level item that I created incorrectly is still
>>> there. Should it go away on it's own as well, it's completely gone 
>>> from
>>> my configuration file now?
>>
>> Now this, I must confess, is something I've yet to do. I need to 
>> remove
>> removed locations from the store. I think I've been delaying because I
>> feel I should wait a day or so to make sure that the adminstrator
>> really wanted to do it, and that's a more complex thing to do.
>
> Why don't you make it go a away when things hit the soft limit as with 
> the
> other stuff?

Marking it as deleted loses information, and would mean that if it 
reappeared, it'd have to be uploaded again. After the delay, it would 
be marked as deleted, and eventually be removed as the soft limit was 
exceeded.

>  That way the admin has a while to discover their mistake... I
> suppose something in the client log the first time or two might be good
> as well. I run logcheck on my Debian machine, and so I see the various
> messages that BoxBackup sends out...

Good. But it's also supposed to email you when things happen.

>
>>
>>>
>>> Do you think people need a way to purposely delete backups? Say they
>>> backed up something they really, really, don't want a copy of? You
>>> seem to
>>> have thought about this carefully what's your opinion? (Something 
>>> like
>>> the
>>> Sarbanes-Oxely stuff in the USA).
>>>
>>> I can see that something like that could really cause problems, but 
>>> may
>>> also be useful if people are managing their own backups (for instance
>>> email retention laws where you probably want it to disappear on the
>>> crack
>>> of midnight the first day you can delete it). I also get the 
>>> impression
>>> you need to be able to hold certain items back from deletion sort of
>>> indefinitely say if the court has ordered you to keep it for some
>>> reason.
>>
>> I have been more interesting in preserving data than destroying it, so
>> controlled deletion didn't really occur to me. But you're right, it is
>> useful.
>>
>> There does need to be a way of retaining data more carefully. When I
>> add the "mark" feature, you'll be able to specify how many marked 
>> files
>> should exist, so you could mark weekly, and say you want to keep at
>> least 6 weeks of marked files.
>
> The Sarbanes-Oxely stuff though would require you to absolutely 
> guarantee
> that something could not get deleted until it was untagged.

Then don't delete it from the client. Things which match files on the 
client are never deleted from the store.

>
>>
>>>
>>> Do you think a config item for max retention time might be useful?
>>
>> Sounds a sensible start.
>
> And again for Sarbanes-Oxely it would have to get nuked immediately 
> once
> it expired, it could not wait until you hit the soft limit.
>
> I suppose you have the wrinkle of what to do with the file if it still
> exists on the client, not backing it up could be counter-intuitive,

Surely the backup system merely reflects the state of the disc on the 
client, and retention is just about what to do on the server when files 
are deleted. If they exist on the client, they exist on the server.

>  but I
> guess you could only keep one copy or something in that case. Actually 
> I
> think that's the wrong way of looking at it, see below:
>
> Maybe if the location had a min-retention-time and a 
> max-retention-time,
> that way you would know the file (older copies of the current file on
> client disk as well) would be kept for at least the minimum time 
> period,
> then all the older copies would for sure get nuked at the
> max-retention-time, the current backup of course would be retained for
> min-retention-time. Kind of like a conveyor belt with stuff falling off
> the end.
>
> The min-retention-time would supersede any garbage collection, you 
> would
> know for sure that file would exist for that time period, if you run 
> out
> of space, then you deal with it...
>
> Again the max-retention-time would nuke the backups immediately after 
> they
> expired, it wouldn't wait for garbage collection.
>
> It might be interesting to have an API where you could programatically
> flag files to be unremovable,

I was thinking of putting something in the bbackupd.conf file.

>  but then, maybe it makes more sense to have
> a regexp in the config file where you could specify files or 
> directories
> as being undeletable. If someone needed to modify things, they could
> modify the config file from whatever program was managing the backup
> stuff.
>
> Finally, to comply with something like Sarbanes-Oxely, that might not 
> be
> good enough, you may need the ability to mark a particular backup(s) 
> of a
> file as being undeletable, but ensure the rest follow the current 
> rules.
> After all, you want to nuke the stuff that is covered by that as 
> quickly
> as possible so it can't be requested by your opponent. However that 
> would
> probably complicate things, so going with the pure conveyor belt method
> would probably be better as a start.
>
> Finally, we would assume that the client has something on the client
> machine that would handle deleting files that had expired, so if the 
> file
> needed to disappear, they would handle that. At that point the file's
> backups would follow the retention rules. Basically limit our scope to
> what we can reasonably control and worry about.
>
> The only other thing that comes to mind is perhaps some sort of 
> optional
> logging when expired files and so on are deleted, that way 
> someone/(some
> program) could be sure that the retention rules were being followed.

Or a command for bbackupquery to check?

>
>>> Is there some way to list how much space you are allowed to have on 
>>> the
>>> drive, along with how much you are using from the backup query 
>>> client?
>>
>> I will add a command to bbackupquery -- I always use bbstoreaccounts 
>> on
>> the server to find out such things, but this is not possible if you
>> don't have that access to the server.

I've done this now. It even draws a pretty little ASCII chart of usage.

>
> I don't think you have really said this, but I see great potential in 
> this
> software being used as a *utility* where someone rents out a server as 
> a
> bit-bucket to others for a monthly fee, having proper retention rule 
> hooks
> would allow it to be used as such in more situations.

Yes, I have considered this. :-)

>
> I don't personally have any real intention of using the software that 
> way,
> but I can see the retention rules being useful to my friends who have
> businesses that I might back up onto my server.

It would all be configured client side, which makes this easier.

Ben