[Box Backup] Housekeeping hogging the server

[Ben Summers]

On 22 Oct 2006, at 23:29, Chris Wilson wrote:

> Hi all,
>
> On Fri, 20 Oct 2006, Per Thomsen wrote:
>> On 10/20/06 2:13 PM, Jan Rychter wrote:
>>> I have one annoyance, though: bbstored does its housekeeping,  
>>> which in
>>> my case both consumes a lot of CPU time and does a lot of I/O, which
>>> slows the server machine down a lot.
> [...]
>> A bit of explanation: The housekeeping process keeps the accounts  
>> within
>> their respective limits for the amount of backed up data. To do  
>> that, it
>> needs to check which files are the oldest, and get rid of them  
>> from the
>> store. This takes CPU and I/O.
>
> I still think the client should do its own housekeeping, and  
> implement whatever policy it chooses on how long to keep old  
> versions of files.
>
> I think that the only protocol change that would be needed is an  
> additional client->server command which causes a file to be  
> immediately deleted (not just marking it as due for deletion).
>
> Of course, the current housekeeping code would need to be moved or  
> copied from the server to the client.
>
> If we want to support low-spec servers, this seems like a good  
> idea, since it would enable the server operator to disable the  
> housekeeping process completely.

I agree with in spirit...

    http://www.boxbackup.hostworks.ca/index.php/0.20_redesign

I'd like the client to control when things get deleted, but use  
reference counting on the server to achieve this. Then housekeeping  
has to do very little to detect files which actually need deleting.

Ben