[Box Backup] How to move store to new machine
E.W. Peter Jalajas
boxbackup@fluffy.co.uk
Mon, 15 May 2006 22:18:55 -0700 (PDT)
Thanks, Ben. Replies below.
Most urgently, at the very bottom:
Exception caught (Common RegexNotSupportedOnThisPlatform (Your platform
does not have built in regular expression libraries.) 1/30), reset
state and waiting to retry...
On the new store I've got:
user@new:~$ dpkg -l | grep -e boost -e pcre
ii libboost-dev 1.33.1-2
Boost C++ Libraries development files
ii libboost-regex-dev 1.33.1-2
regular expression library for C++
ii libboost-regex1.33.1 1.33.1-2
regular expression library for C++
ii libpcre3 6.4-1.1ubuntu2
Perl 5 Compatible Regular Expression Lib
ii pcregrep 6.4-1.1ubuntu4
grep utility that uses perl 5 compatible
What should I do? Do I have to reboot after installing boost? Do I
have too much boost?
Thanks,
Pete
--- Ben Summers <ben@fluffy.co.uk> wrote:
>
> On 14 May 2006, at 16:30, Chris Wilson wrote:
> >
> >> Hmm, I just noticed that my old store raidfile.conf has:
> >> BlockSize = 4096
> >> and my new store raidfile.conf has:
> >> BlockSize = 1024
> >
> > I don't think it makes any difference without RAID, which you
> > aren't using?
>
> No, block size is ignored for non-raid files. But do keep the configs
> the same.
>
> Block size is used by the raidfile lib to make sure that
> it doesn't waste filing system blocks. So you don't "choose" a block
> size, you set it to what your filing system uses as the underlying
> unit of storage.
We should probably be clearer when we talk about raid. I _am_ using
raid (software md raid tools, RAID1 (mirror)), but not boxbackup
userland raid.
So, for the search engines, after some googling, I figured out "howto
determine block size":
user@old:~$ sudo tune2fs -l /dev/hda1 | grep "Block size"
Block size: 4096
user@new:~$ sudo tune2fs -l /dev/md0 | grep "Block size"
Block size: 4096
dumpe2fs also works.
(I'll add a thing about block size to the wiki.)
But, I figured that if I'm gonna start from scratch, I might as well
tune this baby to meet my needs (with a reasonable sample of 8 clients,
85,000 files, and 9GB), so I did:
user@old:~$ for x in 1 2 3 4 8 16 512 1024 10000 100000 1000000
10000000 100000000 ; do echo -n "${x}: " ; sudo find /root/.bu/backup
-size -$x -type f | wc -l ; done
1: 8
2: 21886
3: 39772
4: 47932
8: 60383
16: 69076
512: 82582
1024: 83371
10000: 85227
100000: 85473
1000000: 85492
10000000: 85493
100000000: 85493
So, while my average filesize is 9GB/80K = ~100K, it looks like more
than half my files are under 4k, so I'll keep that as my block size.
(The clients are a small tax firm, property management firm,
environmental consultant, a database guy, a hardware broker, me (x2),
and Mom -- maybe that's a reasonable cross-section of small
businesses.)
> >> I'm too exhausted at the moment to freak out about the fact that I
> >> have to upgrade my clients.
> This decision was made to avoid releasing code which would be used in
> an untested manner.
Sorry, Ben, the freaking was a comment about my own current lifeload,
not at all about the product.
> Really, I would suggest starting again.
> * Stop bbstored. (and make sure all the child processes terminate too
> -- they may delay doing so until the user has logged out)
>
> * As _bbstored, tar up the accounts.
user@old:~$ su _bbstored
su: User not known to the underlying authentication module
user@old:~$ grep bb /etc/passwd
_bbstored:x:1003:100::/home/_bbstored:
user@old:~$ sudo passwd _bbstored
Password:
passwd: User not known to the underlying authentication module
But then, persistence paid off:
user@old:~$ sudo su _bbstored
sh-3.00$ id
uid=1003(_bbstored) gid=100(users) groups=100(users)
I did that on both the old and new store server.
> * Transfer the tar file. (if you don't have enough disc space, use
> netcat to transfer it as it's created)
> * As _bbstored, untar the accounts.
On the new (target) store server, I first made a new store directory:
sudo mkdir -p /media/bu2/backup
sudo chown _bbstored /media/bu2
cd /media/bu2
I ran on the new store server, to get it waiting for the transfer,
based on http://www.securitydocs.com/library/3376 :
sh-3.00$ nc -l -p 4321 | tar xvpz –
Then on the old (source) store server, I ran:
sh-3.00$ tar zcp - ./backup/ | nc -w 3 new 4321
putting the files into a new store area. (I took out the "f" option
because I got an error.) I modified my /etc/box/bbstored.conf to
reflect that new store directory.
That took a few hours, but not too long.
user@new:~$ cat /etc/box/raidfile.conf
disc0
{
SetNumber = 0
BlockSize = 4096
Dir0 = /media/bu2
Dir1 = /media/bu2
Dir2 = /media/bu2
I checked my
/etc/box/bbstored.conf
and
/etc/box/bbstored/accounts.txt
to make sure they were ok.
> * Restart bbstored on the second machine.
user@new:~$ sudo /usr/local/bin/bbstored
user@new:~$ ps -ef | grep bb
1005 22047 1 0 23:27 ? 00:00:00 /usr/local/bin/bbstored
1005 22048 22047 16 23:27 ? 00:00:01 /usr/local/bin/bbstored
I then did:
user@new:~$ sudo crontab -e
and inserted:
#restart bbstored if it isn't running
*/15 * * * * if [[ `ps -ef | grep bbstored | grep -v "grep bbstored" |
wc -l` -lt 2 ]] ; then killall bbstored ; /usr/local/bin/bbstored ; fi
I then sent a mail to all my clients saying that they are not backing
up at the moment, and that I need to upgrade their client backup
software ASAP. (Maybe I shoulda woulda coulda done them one at a
time...)
So, now I'm setting up via Remote Desktop Connection one of my clients,
following the Upgrading instructions at
http://boxbackup.hostworks.ca/index.php/Compile_on_Windows . The
install went fine (first make a backup copy of your C:\Program
Files\Box Backup\bbackupd.conf, and make sure that the zip Extract
function actually worked (you should see "Replace...?" messages); I
changed the CommandSocket to "pipe" because that was in the
bbackupd.conf that came with 0.10, and I extended MaximumDiffingTime to
40, but I'm not sure why.)
And so, it's trying to connect to my new server, but now on the client
I get:
Exception caught (Common RegexNotSupportedOnThisPlatform (Your platform
does not have built in regular expression libraries.) 1/30), reset
state and waiting to retry...
On the new store I've got:
user@new:~$ dpkg -l | grep -e boost -e pcre
ii libboost-dev 1.33.1-2
Boost C++ Libraries development files
ii libboost-regex-dev 1.33.1-2
regular expression library for C++
ii libboost-regex1.33.1 1.33.1-2
regular expression library for C++
ii libpcre3 6.4-1.1ubuntu2
Perl 5 Compatible Regular Expression Lib
ii pcregrep 6.4-1.1ubuntu4
grep utility that uses perl 5 compatible
The server /var/log/box shows this client (and all the others) logging
in, and for this client, show listing directories, setting store
marker, session finished, but no mention of any problem.
What am I missing?
Thanks,
Pete
> It seems like "something funny" has happened. This should all just
> work.
>
> Ben
>
>
>
>
> _______________________________________________
> boxbackup mailing list
> boxbackup@fluffy.co.uk
> http://lists.warhead.org.uk/mailman/listinfo/boxbackup