[Box Backup] child bbstored process ramping up to 100% cpu during bbackupd sync?

[Ben Summers]

On 31 Jan 2005, at 21:19, reticent wrote:

> Ben Summers wrote:
>
>>
>> On 28 Jan 2005, at 22:45, reticent wrote:
>>
>>
>> What you are seeing is a patch being received by the server, and 
>> applied. The 20-30kb of data is the patch, then there's a bit of 
>> computation required to apply the patch to the existing file, which 
>> overwrites the original file.
>>
>> What is the usage pattern of files on the client? Are they static? 
>> Are they being constantly modified? Are files being added regularly?
>>
> The files are, for the most part, static. There are only a few hundred 
> updates/additions per day.
> They are also very small, ~1-20kB
> No modifications are made to them

What's the directory structure like?

>
>
> The quoted speed was during a backup, i'll have to take some time to 
> do some debugging as somthing is definitly not right here.
>
> I'm getting the feeling that it might be related to the amount of 
> directories that are being delt with, the backup completed (30+ hours 
> later for 1.6 gigs..) and i'm seeing the hk process run and eat up 
> 100% cpu when checking backup data for host mentioned in the original 
> email.

Yes, housekeeping can take a while for large directory structures. I 
will be addressing this soon.

>
> Looking at the lsof output for the file, it seems to spend approx 6 
> minutes opening/closing a single file (that is exactly 234k, for some 
> reason all the files i've noticed it working with are exactly 234k)

Interesting. It's probably the directory file you see being re-written 
repeatedly.

>
> I'm going to do some testing with faster machines in a seperate 
> environment.

You are using the latest version, aren't you? This had a fix for large 
directories which may help.

Ben