[Box Backup] Housekeeping hogging the server
Wed, 25 Oct 2006 14:40:45 +0200
>>>>> "Ben" == Ben Summers <firstname.lastname@example.org>:
Ben> On 25 Oct 2006, at 12:23, Jamie Neil wrote:
>> I'm in the process of setting up a new server for backing up the
>> other servers in our rack, and I was planning on using Boxbackup.
>> Unfortunately it looks like it's not going to be practical due to
>> the extremely high CPU loads that we are seeing.
>> Server is a 2.8GHz Xeon with 1Gb RAM and RAID 1 SATA. Currently it's
>> backing up 7 other servers with a total of 50Gb. None of the
>> accounts is even close to their soft limits and yet the CPU has been
>> pegged at close to 100% for the last 2 days.
>> I saw this behaviour before when we were experimenting with
>> Boxbackup and it turned out to be a couple of folders with thousands
>> of small files in them. Once we removed them then the problem went
>> away. I suspect the same problem this time as one of the servers is
>> storing mail in maildirs (Cyrus) and may have several folders
>> containing a few thousand files each.
>> I assume that this is a housekeeping issue because there are
>> currently no clients connected.
Ben> Yes, this is most likely a housekeeping issue. The client is also
Ben> not happy with directories with thousands of files in them. I do
Ben> want to address this in a future version, it has come up a few
Incidentally, this is also my use case (I started this thread about
peformance issues). I use boxbackup to backup my E-mail storage, which
is in nnml format -- with directories reaching over 10,000 files