[Box Backup] 100% CPU on big directories

[Jamie Neil]

Hi all,

I've been testing the latest release candidate on a few of our servers,
and I've noticed a problem related to one server where we have a
directory that archives blocked spam. This directory has now reached
1.5GB in 150,000 files and when boxbackup tries to back it up the CPU
utilisation on the bbstored server hits 100% and stays there (we've left
it running for the best part of 2 days and it never finished).

The problem seems to be the number of files in the directory (it's flat)
because we have other targets on the same server which are much bigger
but only raise the CPU utilisation to 20-30%.

I noticed that there was another post about a similar problem:

http://lists.warhead.org.uk/pipermail/boxbackup/2005-January/001067.html

but no firm conclusions.

Both the bbstored server and the server with the large directory have
plenty of horsepower (3GHz Xeon and Dual 1GHz Xeon respectively),
neither seem to be running out of memory, and they are talking over a
100Mbit switched LAN, so I assume that this is a limitation in boxbackup.

Yes, I know it's not clever to have a flat directory with 150,000 files
in it :), but I've noticed the effect to a smaller degree on directories
with only 1000 files. Does anyone know if it's fixable?

-- 
Jamie Neil | <jamie@versado.net> | 0870 7777 454
Versado I.T. Services Ltd. | http://versado.net/ | 0845 450 1254