[Box Backup] managing disk space

Wed, 21 Jul 2004 09:45:44 -0400 (EDT)

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--8323328-959055098-1090417544=:3742
Content-Type: TEXT/PLAIN; charset=US-ASCII

Here it is...

You need to run it as a cron job using a period of half the period of the
shortest time span in which you want to detect a dropout, or less (twice
the frequency).  This is because of the Nyquist sampling theorem. If you
do not do this, you may get false positives.

The only problem I've noticed is that the Perl module I use to read the
logs crashes after a reboot on my Debian machine until the logs are
rotated.

You need to look at the Perl file to see the file locations it uses,
basically just /etc and /var/lib, but you need to create the
/var/lib/dropout_tracker directory manually.

This code should work on Debian Linux, Solaris and the various BSD's,
although I have only run it on Debian. The limiting factor should be the
Perl module that decodes the syslogs.

I have included the Perl script, an example configuration file and the
file that is created in /var/lib (however that file is recreated each time
the script is run, so it's for informational purposes).

The dropout period can be more or less any arbitrary period, the syntax is
limited by the Perl date module I used to decode it.

If you have any questions, feel free to ask me about it.

Rick

On Wed, 21 Jul 2004, Nick Knight wrote:

> Hello Rick,
>
> If you could send it over that would be great.
>
> Regards
>
> Nick
>
> -----Original Message-----
> From: boxbackup-admin@fluffy.co.uk [mailto:boxbackup-admin@fluffy.co.uk]
> On Behalf Of rprice@freeshell.org
> Sent: 20 July 2004 19:52
> To: boxbackup@fluffy.co.uk
> Subject: RE: [Box Backup] managing disk space
>
>
> I have such a script written in Perl, but it is actually more general
> than
> that. I would be happy to send it to you if you want. I run it on the
> server.
>
> However, whenever the client cannot backup, it logs the error on the
> client, so you can check there as well.
>
> Rick
>
>
> On Tue, 20 Jul 2004, Nick Knight wrote:
>
> > There was one thing I am trying to figure out which I was hoping maybe
> > in some other tools. From the server end perform a report that all
> > clients are sucsesfully backuping up or not - plus other details, is
> > there enough info at the server end to do this?
> >
> > Thanks
> >
> > Nick
> >
> > ________________________________
> >
> > From: boxbackup-admin@fluffy.co.uk on behalf of Ben Summers
> > Sent: Tue 20/07/2004 10:41
> > To: boxbackup@fluffy.co.uk
> > Subject: Re: [Box Backup] managing disk space
> >
> >
> >
> >
> > On 19 Jul 2004, at 21:13, Nick Knight wrote:
> >
> > > Hello all,
> > >
> > > Quick question on how to keep account sizes down. Out of the box the
> > > system just puts in data, is there a way to get it to remove deleted
> > > files after 30 days for example?
> >
> > When the space used reaches the soft limit, the housekeeping process
> > will start deleting older files to keep the space used below this
> > value. It won't grow indefinitely! If bbackupd attempts to upload a
> > file which would take usage over the hard limit, then the file will
> not
> > be accepted.
> >
> > Just set soft and hard limits appropriately, and everything will be
> > fine.
> >
> > There will be more control in the version after the next version. I'm
> > just putting the finishing touches to storage of older versions on the
> > server as patches from the latest version (all works, just working on
> > the "fix broken account" code), and then will write up a spec for what
> > I'm going to do and post it here for comments. This will essentially
> > emulate what you can do with tapes, but use less space.
> >
> > Ben
> >
> >
> >
> > _______________________________________________
> > boxbackup mailing list
> > boxbackup@fluffy.co.uk
> > http://lists.warhead.org.uk/mailman/listinfo/boxbackup
> >
> >
> _______________________________________________
> boxbackup mailing list
> boxbackup@fluffy.co.uk
> http://lists.warhead.org.uk/mailman/listinfo/boxbackup
>
> _______________________________________________
> boxbackup mailing list
> boxbackup@fluffy.co.uk
> http://lists.warhead.org.uk/mailman/listinfo/boxbackup
>
--8323328-959055098-1090417544=:3742
Content-Type: APPLICATION/x-gtar; name="dropout_tracker.tgz"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.56.0407210945440.3742@server>
Content-Description: 
Content-Disposition: attachment; filename="dropout_tracker.tgz"

H4sIAFxw/kAAA+1abVPbVhbOV/wrTok7QNfFNhhoRJOpsQVoF2xq2WEz292M
kK5tDbKkSjKEZtjfvs+5V7KFMYU0KZ2d6HSKbemec8/Lc16uFCcKwmCavE8i
y74UUfXFn0A10N7ODn/W93Zq+c+MXtRrtZ09LKpt1V/U6ju13foL2vkzlFmk
aZxYEdGLKIxcWzy87rH7mSHZ5/8JOQvxb701vzgGnhj/vb3GHv5uc/x36ntF
/J+DlsW/FwTJl9yD/bHbaDwcf2CD8393b3drb3cP8d9twIP0LE78yuOvhbGI
rkSkKft+Sn9Vx8FEVO2reIoL/BkBE6W/WtmCvjgtzX8RBrGbBNHNl9nj0fzf
qi3kP740ivx/DlqIf5HiXxkty3/dTyJXxF9sj8fyv7G9vTD/7dV2tov8fw6q
LgBg0x6ONmPrSlTrm/K/an8q6O9Tn6hOtS2tXtd26rSFoFWrpXvM8GUi/jh7
6IFtr3oYuYqlwSy1H7TtrQdZoC54dqrnwgGPR1t1qm9r2zWt9ur3tQRXY5Gr
rtUaGVf76yiFi/m/xL+fvcdj+b+3k+X/Lv7H+X8L3+tF/j8HvfSC0dD1hBZG
wSiyJtowEr9OhW/faPFNjJs0EXFsjbDAShIR+aWX9A8hQrq2EntMgU8HwYcD
YGUalqpXVlQFS1VxahcXMYZI4Wh1x7qJtTbKArWsKKKhhR0dSgK6kJzaSTBy
fY1aniv8hIw2pV6sV6gnLKd6HrmJeEx+azyNoNKThW89IlxyO9ruOJhGsdZz
7UtiT6kT0v1tDvGbuLS4ceLasYZbieVJFord3wRNQy+wHOHAg6fWJS5OI0G2
F0DuSCQxxTe+jbuLivgJlNiTFpqLWzMLJe5E0LWbjMlKgolrK5mxxjfHUeBj
b168vJ49lv+h9/kY+/3nP/XGFob99PlfAyd/zv/dRpH/z0Ivv6lO46h64frV
UEReaeE3fX8NuJYNR6P7yKhcEdq1api13Wot166Vt0j/EFK5VIKIVhDeRO5o
nNC6vSFZkH2cUmdy4Y/4e/lePYLgRw7xZoyss8UwiEZi0xfJG2p6HkkJMUUq
CZ1NltwTDjIuci+miYtyZPkOQQK5PikR8grssaIbgrhJXFHJEkTyEyZByCRw
3KFrWyyiQhYSE9ZPXFQ8h1AZr1yHc2hsJfgjIMbzgmvXH5Ed+I7LTDEzsSCR
aKwV03cLusUUDDOl7MARNAH4YExiQVmWa10EV3wrdVUqhsgPUFFEBWvcmDxI
ZEH5rX1nQS9sa3sWCkO0+aAy2DTnlkwZWOtMoeBMn5kWM70+XZ+ZjLlelNrs
BPZ0gsJsZdGrIjAB7kQ0wZgWuZYXz0IwkyMjKNlzJs1N7QhXSuAVvoXyCP1y
aPOD+S0OSs5EX0kLohjb39CFYDDJWit8B1cF4wbqTIJEkPIVAOlAz6ucekMs
UN6Jg2FyzXhKsUZxKGyGGphdhmDEIPMV3OJYWgEx/WPDJLN72D9v9nTC97Ne
963R1tt08A43dWp1z971jKPjPh13T9p6z6Rmp42rnX7POBj0uz0TUlabJnhX
5a1m5x3p/zzr6aZJ3R4Zp2cnBsRBfq/Z6Ru6WSGj0zoZtI3OUYUggjrdPmSc
GKdGHwv73Yrc+D4jdQ/pVO+1jvGzeWCcGP13csdDo9/h3Q67Pchp0lmz1zda
g5Nmj84GvbOuqRMb1zbM1knTONXbm9AAu5L+Vu/0yTxunpzctRViuucdvccG
5E2lAx16Ng9OdN5Mmto2enqrzzbNv7XgQKh4UoEc80xvGfgKn+iwqNl7V0ml
mvrPAyzDTWo3T5tHMHD9Uc8gPK1BTz9lxeEOc3Bg9o3+oK/TUbfblh439d5b
o6Wb+3TSNaXTBqbOqrSb/abcHELgMyzA94OBaUjvGZ2+3usNzvpGt7OBYJ/D
O9CzCea2dHO3wwZLzOjd3jsWzL6QcajQ+bGO6z32rPRYk11hwnOtfn4ZdoQj
2aq5rdTRj06MI73T0vl+l+WcG6a+gbAZJi8w5NaAAnYdSMM5XNCMA3V4F8QV
GVYyDqnZfmuw8ulyAME0UthI17WOU8erTJBZZEdumCDNEsHpdj1GxliUDqjp
YCrHopgzdWyFIRZwxrm8boTc9NWY5PpYemV5kMxjGNIXA12aqXIPbi6/Tl10
GKzAEkwHvmNFTol7iokZTDPlrvvypgitCEVK3uxDvqadWagRbVzblxflzzwP
uM4C5PkFpkJ3wnVNcPVLd+tzb83MszwRJZizUWItlFhl7QjFiMvrteUm7AJV
TXAxs4zW08KKWjZ0R3L+3MiJdjEiTywbnklrvJdk28iqdZ8bNYxuginZlq86
a6LKL7SJZU+EdK5wXL75IqkJWUZvsOgDjk/IF+QuaQTlWK2c44gha4YZ2LUu
PBmFMoB7aBy9PzSQ369prSoSe9k5dW2/VDb7zb4+XylHafdicfXyJwLMLysE
WFcXVqymsZNe9uHlkRdc4OOOnowwSJqE5whgb+pDDl9KWWORTMPUYlwIAFG2
XW1ZodXQdVYrqA1H79tN/bTb2S+l3lnla0bnsIvbqyam1QQdNVOoFQmoDphG
aEQ4UMgESNJYY1xJkwM5I2SQvCC45AEBLV9B7sy9ChKOXRaICqVHQdU5GWyp
ENyZxuO0Obs+Tw2ya5ekMbSei9MG1HOHtC7QePOX6WOJ+2PeMlS31crqz1P0
QSi2TwOfHcpmSKkKiNNIzQcSj2vfxmvfaPTtZBMeWS/f3Va2cS+IhUw4/iU+
uMl+6bZUgsICyKf1H3MsbzKd7HEQQmv+ijiPMCNwFkw4O5FeHlAZy5tsVvU/
L6vMt+KLD4na5fYu54Vn+Zf3uH6J/1ZezphyQ0FOZeDKm6ac6+X0iF4pp4Gp
lGeH9Eo5PZ7jpgrSBlAXh56brFe1aqX8vrKTOqVsY6wZDrlMYcl9sH6v8tIB
nNZzG6DZndPrN7RWW9uYuQeK2Ng3VbOCisu4GKpPdRRFMYgiTFBWLDNdrpTc
jKH1nz6yVe9l4n/MDLz9mFnI35Q5t7eV9Xu6LnfA3L6NDRlvCYM8/PZLKmkw
fo4QXJk6Tj53pOr8+NKRJ3nBKcKRU/ieF5cNjqG0Zo6p+V0JqRWFp5VSaeVB
OK3cwZKc9CUsVm5zTHkkrSzC6A4PMy0AaGUZeridxEL4D2OmwaGGtGnoLFQX
KCC7pZRP9jSKYJB3I6dkHAjQ2lR9uHZjodRVSqq67tCysMsa8/iyOTqY4VPW
z9GUbjPz3VP5/lX79+uZ35SzVe7O6w3NAQDn3ZLwcO1+ucMs1MHsxLX8CZUu
19DWeKEMr4TmQ/XvVkF8oCKnJgeRjT/D9OCTdYUKph5kqkRwaj5agc24z4Cd
uYXWL8VNTN/OHDZLAV4he3wEGN1p9t+/8cX1HIGyIERiKMtJfS2tSxhJWDqv
4hdPN1ycZhJZwockK9Gq+Jy5QDifrHDW5sJA/tTzMN8zMJEpITdIe+pZEWAJ
ALrJGk6khCbuCw+nOGzB0C6j30rglccBfCSP/NMo5MYBNPuMZ86eWempqIX4
mOURJFRY1b5U8TWpoiaNuFX5w03X45jd0KUfXM+fIGROdeMZhC+EbfF4dZ0e
OaW7AYS5x1U+PQ3yM4TP4piNyWkcHynAUkBphcvQkFUa88NbPr1jYI3kDEqB
jfRHDZan6xyqKlnZuAs+CGP1M4fR6/9SNVOqmu6mAsyeSkVIX93FMM838moi
+MFDpkTKPUvS15+Q2vsps9RvFnF6M5cm3bmy8knlYiYoE4+qzpU9/TorIWnG
muxhmdrs72vLT+TDZfkkKnbjBMjldrTYilBH3uSqxOoGP6BYZ2VXVMJ8/qwl
nzTPtMtNXrmGqErybFOeuXI/uZ2EEYbnXJGk1XvvHjJvZ+8afvHlmEsYxEN+
dCKdw51QFrJ81ZoNrZXZcRDNlh9vyQH1ScXsbrakM/DvZYtKss9JMOme+47J
lmvZau0TgKeV895buZ2jbEmbkkeIscBpgWMsBKeA5d/Mjw0y78UHWwj1BFS4
EX03y8jvVEpyGVWnmueK1R8M1udFq/RZFWZhBn8693bGLctTboq/V5+yWfjp
srdk7btfItrq+JtiQ71w0mQxUJmfbrSxnytnM5xJHNie4OcFIX4sO8ceur4b
j4UDiM5PayV1VPur34l8TfSkJyOfucej//5ntzH/9z/b/P6vvr29Vbz/ew56
tAcvfx9dr72qNepbjdr2vdfMD7+lZ57aD9tbO3/4lfwdSdu7tYclPfYvB9KX
+6kdr3549Qnv7//qmBVUUEEFFVRQQQUVVFBBBRVUUEEFFVRQQQUVVFBBBRVU
UEEFLaP/ARwvjLIAUAAA

--8323328-959055098-1090417544=:3742--