[Box Backup] How to move store to new machine
E.W. Peter Jalajas
boxbackup@fluffy.co.uk
Tue, 16 May 2006 12:56:48 -0700 (PDT)
I'm still not sure if all the new file transfers was a good thing or a
bad one, but this info may make sense to you guys.
Here is the
sudo bbstoreaccounts info <acctno>
report that I set up to get emailed from my old server to me every two
hours:
Account ID: <acctno>
Last object ID: 29249
Blocks used: 831017 (3246.16Mb)
Blocks used by old files: 152442 (595.48Mb)
Blocks used by deleted files: 22203 (86.73Mb)
Blocks used by directories: 2050 (8.01Mb)
Block soft limit: 896000 (3500.00Mb)
Block hard limit: 1536000 (6000.00Mb)
Client store marker: 1147615061000000
user@new:~$ sudo bbstoreaccounts info <acctno>
Account ID: <acctno>
Last object ID: 41765
Blocks used: 846946 (3308.38Mb)
Blocks used by old files: 168183 (656.96Mb)
Blocks used by deleted files: 22237 (86.86Mb)
Blocks used by directories: 2254 (8.80Mb)
Block soft limit: 896000 (3500.00Mb)
Block hard limit: 1536000 (6000.00Mb)
Client store marker: 1147803312000000
So it looks like the new setup (client and server) found many more
files to backup, and so all my stored deleted files were deleted to
make room. No why did I have so many new files? Either because regex
exclusions is not working (and not reporting an error), or I'm picking
up new directories or files somehow. With 192MB freeboard between
Blocks Used and the soft limit, I think we are pretty sure that we're
getting all the files that we're supposed to get. I need to make sure
that I get a new version of their 180MB quickbooks file tonight. I'm
guardedly optimistic that we're ok. On to the next client...maybe.
(For the search engine record,) I recommend making the hard limit about
at least twice as large as your largest file (check your quickbooks
files, Outlook and Outlook Express mailboxes, databases). I have
several clients with GB+ mailboxes. So, you need that space between
soft and hard limits for those files to come in, be handled correctly,
and then old files are deleted. Do NOT do what I originally did and
make the hard limit arbitrarily 25% or 50% larger than the soft limit;
the gap needs to be a multiple of at least one or two times your
largest expected file for that client account. Depending upon your
specific conditions, don't be afraid to have a total sum of hard limits
across all of your accounts which add up to much more than your server
can hold because that storage (the amount between the soft and hard
limits) is very rarely used for any length of time (kinda like
overbooking an airplane, the chances of everyone showing up at the same
time is quite small, and I think in this case, I'm hoping the box
backup will simply back off and try to upload the second large file
later so that everyone will get on board soon enough).
--- "E.W. Peter Jalajas" <pjalajas@tebuco.com> wrote:
> Hi Chris,
>
> Funny to see myself typing this because I really have no idea, but I
> think the _bbstored user and PAM are fine. He is odd because he is
> created without a password (as instructed), which I think is secure
> because no one can become him except by doing sudo su _bbstored
> (which
> is simple to do when needed).
>
> Re the permissions, I have no idea either except for the netcat tar
> deal I did. Maybe this confirms that I'm ok:
> drwxr-xr-x 2 _bbstored users 4096 Mar 1 16:44 0e
> drwxr-xr-x 2 _bbstored users 4096 May 16 12:55 0f
> The old directories has the same perms as the new ones.
>
> I'm a little nervous at the moment about so many new files being
> created.
> -rw-r--r-- 1 _bbstored users 135 Feb 9 12:14 o23.rfw
> -rw-r--r-- 1 _bbstored users 67015 May 16 12:49 o24.rfw
> -rw-r--r-- 1 _bbstored users 135 Feb 9 12:25 o28.rfw
> -rw-r--r-- 1 _bbstored users 58343 May 16 12:49 o29.rfw
> -rw-r--r-- 1 _bbstored users 135 Feb 9 12:35 o2d.rfw
> -rw-r--r-- 1 _bbstored users 38055 May 16 12:49 o2e.rfw
> -rw-r--r-- 1 _bbstored users 135 Feb 9 12:37 o30.rfw
> -rw-r--r-- 1 _bbstored users 9703 May 16 12:50 o31.rfw
>
> user@new:~$ date ; du -s /media/bu2/backup/<acctno> ; find
> /media/bu2/backup/<acctno> -type f | wc -l
> Tue May 16 13:26:05 EDT 2006
> 3360376 /media/bu2/backup/<acctno>
> 23788
> user@new:~$ date ; du -s /media/bu2/backup/<acctno>; find
> /media/bu2/backup/<acctno> -type f | wc -l
> Tue May 16 14:45:41 EDT 2006
> 3396856 /media/bu2/backup/<acctno>
> 31050
> (this redaction/obfuscation thing I'm doing is a big pain in the neck
> waste of time, isn't it?)
>
> And I can't see any files that are dated after Friday... I'll have to
> wait and see because of the weekend and the flooding we had yesterday
> Monday, maybe they didn't create any files yesterday... Heck, maybe
> this all fixed something that was broken before.
>
> I didn't, but should have, run
> /usr/local/bin/bbstoreaccounts check <acctno> fix
> on all my accounts before I did the tar netcat. That should be added
> to the wiki. (I think I have at least 3 wiki edits I need to make
> now.)
>
> user@new:~$ grep -i user /etc/box/bbstored.conf
> User = _bbstored
> which was set during the server config with
> /usr/local/bin/bbstored-config /etc/box hostname _bbstored
> I believe.
> http://boxbackup.hostworks.ca/index.php/Configuring_a_server
>
> Thanks again,
> Pete
>
> --- Chris Wilson <chris@qwirx.com> wrote:
>
> > Hi Pete,
> >
> > > Sorry, I'm still not sure what you're getting at re the _bbstored
> > user.
> > > Can you paste the lines that you think are odd?
> >
> > OK, I missed the fact that you were running the commands on
> different
> >
> > servers. On the old server, _bbstored has uid 1003, and on the new
> > one
> > 1005. It shouldn't matter, but make sure that your restored tarball
> > has
> > the correct permissions to allow the _bbstored user on new to write
> > to it.
> >
> > It may not matter either if you're decommissioning the server, but
> > this
> > worries me:
> >
> > > user@old:~$ su _bbstored
> > > su: User not known to the underlying authentication module
> > > user@old:~$ grep bb /etc/passwd
> > > _bbstored:x:1003:100::/home/_bbstored:
> >
> > This implies that your PAM setup is broken on the old machine. I
> hope
> > this
> > works on the new machine?
> >
> > Did you already try fixing the store on the old machine before
> > copying? If
> > not, please try that.
> >
> > > (Do I recall reading somewhere that you start it as root, then
> > _bbstored
> > > takes over, or was that some other project that I'm thinking
> > about?)
> >
> > I'm not sure. There is a "user" option in the server config, but I
> > haven't
> > used it, so I don't know if it works.
> >
> <snip>
> _______________________________________________
> boxbackup mailing list
> boxbackup@fluffy.co.uk
> http://lists.warhead.org.uk/mailman/listinfo/boxbackup
>