[Box Backup] Housekeeping taking long?

[Ben Summers]

On 24 Jan 2005, at 19:46, Imran wrote:

> Well, I just checked the backup server, and housekeeping finished 
> about 10
> minutes ago.  about 11hours:
>
> Jan 24 00:12:36 backup bbstored[20595]: Starting daemon (config:
> /etc/box/bbstored.conf) (version 0.09)
> Jan 24 00:12:36 backup bbstored/hk[20596]: Housekeeping process started
> Jan 24 00:12:36 backup bbstored/hk[20596]: Starting housekeeping
> Jan 24 11:30:22 backup bbstored/hk[20596]: Account 0x00010101, removed 
> 198451
> blocks (27205 files, 0 dirs)
> Jan 24 11:32:42 backup bbstored/hk[20596]: Account 0x00020202, removed 
> 26139
> blocks (144 files, 0 dirs)
> Jan 24 11:36:36 backup bbstored/hk[20596]: Account 0x00030303, removed 
> 48
> blocks (0 files, 24 dirs)
> Jan 24 11:38:00 backup bbstored/hk[20596]: Account 0x00040404, removed 
> 134325
> blocks (142 files, 4 dirs)
> Jan 24 11:38:00 backup bbstored/hk[20596]: Finished housekeeping
> Jan 24 11:39:00 backup bbstored/hk[20596]: Starting housekeeping
>
>
> I guess there were a lot of files to be removed, for 10101, and 40404. 
>  Does
> it print a summary after it finishes all the file removal?  or after 
> each
> account?  11 hours for 10101, and removed 200k files, but 4 minutes 
> for 40404
> and removed 134k files.

Those figures are in blocks of about 4k, not files. The files removed 
are in brackets afterwards.

>
> I'm going to check back later to see how long housekeeping takes for 
> this next
> iteration.

Probably about the same time. In account 10101 you have an awful lot of 
files (as you can see by the last used object ID in the account stats 
from your last message) and it is taking a while to read them all in 
and work out what is to be done next.

This isn't terribly helpful behaviour. Maybe I should keep extra 
meta-data to speed up housekeeping when accounts are large? I shall 
think about this, and come up with a solution.

Ben