I have a SLES 11 server that is accepting syslog data from a remote system (thanks to assistance I got on this forum). The data that system is sending to the SLES 11 system amounts to between 5GB and 6GB per hour. The file it is putting the data into, on the SLES 11 system, is /var/log/audit/taudit.log.
I’ve created a file in the /etc/cron.hourly directory called syslog and I did a chmod against that file in this fashion:
(in the /etc/cron.hourly directory)
chmod 755 syslog
Logrotate conf files are not executable by themselves so placing one in
/etc/cron.hourly may result in cron trying to run it but there is no
executable code in there so you’re probably just building up errors
every hour in /var/log/messages as cron fails to run what is not valid.
If you want to have hourly log rotation you could perhaps do this
most-easily moving the /etc/cron.daily/logrotate file into
/etc/cron.hourly which would make the logrotate script (which is
executable and has valid code in it too) try to run. You would also
need to move the current ‘syslog’ file in /etc/cron.hourly to
/etc/logrotate.d (be sure to not overwrite the file you probably already
have in there with that same name).
Another way you could handle this is to just copy your ‘syslog’ contents
into /etc/logrotate.d/syslog as a new section along with the other
sections, and then still set logrotate to run hourly as mentioned above.
Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
Well, I could try that and I’m perfectly willing to try it (and I really appreciate the help you’re giving me) but I don’t want everything to rotate hourly, just that one log file. Is there any way to accomplish this without rotating every log file hourly?
Sure, a couple of ways. Before getting into that though, everything
other than what I’ve already suggested will probably be more work and/or
less-clean from a maintenance perspective. What is the drawback to
rotating hourly? If you are concerned about the number of files (24
hours * 30 days * 12 months… that kind of thing) then you’re probably
forgetting that logrotate does not force a new log file every single
time it tuns. The logrotate definitions control how often that happens,
such as when a file gets to be a certain age, or a certain size, or
something. Tuning in that area is already done and could just be done
more to prioritize based on hourly stuff for this new section.
Going back to other solutions…
First, don’t use logrotate but instead put a script in /etc/cron.hourly
that renames the file and restarts the service (or at least sends a -HUP
signal) for you. That’d be really easy, but it doesn’t use logrotate
which you may want to use for some reason.
Second, copy the logrotate script from /etc/cron.daily to
/etc/cron.hourly and then modify it to ONLY point to one logrotate
configuration file which happens to be the one that you are going to
setup for this one syslog file. Now you have two places to maintain
both the cron/logrotate scripts as well as the logrotate configuration
files.
Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
I took the suggestion from your first reply and did that (mv the syslog file from cron.hourly to /etc/logrotate.d and move the logrotate file from cron.daily to cron.hourly). But that has produced another problem that I need to solve (a problem you mentioned in your second post); after it rolls this audit file, compressing it and renaming it with the date as part of the name, it doesn’t “re-initialize” the file it just rolled. It leaves that file there, which is not what I want. I want that file deleted so that a new one can be started from a new file (a file that starts out as empty).
Now, I noticed that in that “syslog” file that I have now moved into the /etc/logrotate.d directory, there is a line that instructs that /etc/init.d/syslog reload should be run. I’m thinking that what I need to do is to create a shell script, maybe also putting it into the /etc/init.d directory and in that shell script have the following command:
service syslog stop
rm /var/log/audit/taudit.log
service syslog start
and then in that syslog file located in /etc/logrotate.d directory, change that line that says /etc/init.d/syslog reload to
/etc/init.d/syslogt reload and name my shell script located in /etc/init.d syslogt. Would that work?
See it isn’t just a metter of rolling the log file. I also need to delete the old one while initializing a new one. The whole point being to reduce disk space consumption. Otherwise, I’ll run out of disk space in just 2 days.
That should happen automatically since otherwise nothing about “log
rotation” is really happening. You should not need any other script
besides the ‘reload’ piece that is already there if you are using syslog
to write your file. Putting another script in /etc/init.d would be the
wrong way to go since that’s where service scripts go and what you’re
proposing is not a service (/usr/bin or /usr/local/bin or something
would make more sense).
Why is it not working as you expect? I’m not sure. You could
potentially use the pre/post scripts to move the file and then reload
the applicable service (assuming it’s syslog, or works like syslog with
a ‘reload’ argument) and then rotate your renamed file in case this is
some issue with the file handle held by the application not letting
rotation work properly. You could also add something like the following
to the postrotate script which is probably much simpler/better:
echo > /var/log/audit/taudit.log
Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
[QUOTE=dwoeltje;4081]I took the suggestion from your first reply and did that (mv the syslog file from cron.hourly to /etc/logrotate.d and move the logrotate file from cron.daily to cron.hourly). But that has produced another problem that I need to solve (a problem you mentioned in your second post); after it rolls this audit file, compressing it and renaming it with the date as part of the name, it doesn’t “re-initialize” the file it just rolled. It leaves that file there, which is not what I want. I want that file deleted so that a new one can be started from a new file (a file that starts out as empty).
[/QUOTE]
You can and should test your command, which you put in cron.* stand alone in the shell.
Watch the output. You may increase the verbosity.
Did you set dateformat? Otherwise dateext will only by YYMMDD, which is to little information for hourly rotation.
man logrotate:
[code] dateext
Archive old versions of log files adding a daily extension like
YYYYMMDD instead of simply adding a number.
dateformat
Specify the extension for dateext using the notation similar to
strftime(3) function. Only %Y %m and %d specifiers are allowed.
The default value is -%Y%m%d. Note that also the character sepa-
rating log name from the extension is part of the dateformat
string.
your postrotate script seems to be set to remove uncompressed rotated files(“rm /var/log/audit/taudit.log-???”) - do you actually see those? Or is that a simple clean-up measure in case someone uncompressed old logrotates and leaves them in that directory?
It seems to create two compressed files; one with a .bz2 extension and one without a .bz2 extension. I don’t need two copies of the same thing. So I keep the one with the .bz2 extension and delete the other one. Since it appends the date, I have no easy way of determining (in an automated fashion) what it will append (I’m not a programmer, so while someone else might be able to do this, I cannot), so I was left with trying to figure out a way to delete all files that begin with taudit- but not those that end in .bz2. Assuming that the wild card placeholders would work in Linux the same as they do in DOS (* being a wild card for an unlimited amount of anything and ? being a wildcard for anything but only one character of anything) and given that the date appended will always be eight characters, I simply had it delete taudit-???. That deletes what I don’t want and leaves the files with the .bz2 extension.
But I’ve run into another problem. Because I’m rotating hourly and there doesn’t seem to be an option in logrotate called timeext, the first logrotate for a given day works just fine. But all logrotates that come after that fail because they will have the same name. So, in addition to a postrotate entry, it would seem that I also need a prerotate entry, one that would change the name of the taudit-???.bz2 file to taudit-???.HH:MM.bz2 (with the time being where HH:MM is located). But that creates a problem:
I don’t know how to go about getting the time inserted into the file as part of an automated process. Since all other files in the directory would already be renamed taudit-???.HH:MM.bz2, there would only be one file that would match the filename pattern of taudit-???.bz2, so I would hope that I could simply do something like mv taudit-???.bz2 taudit-???.$TIME.bz2 (or whatever I would have to do to get the time inserted in the filename.
No. According to the man page only %Y %m and %d are allowed (and you are right, that is not enough for hourly rotation; I get one file and all other attempts for that day fail because there is already a file by that name there). It doesn’t say anything about allowing %H %M %S (and that would have two M’s being used, one for month and one for minutes so unless it distinguishes between the two using case (uppercase versus lowercase), how does it tell them apart? I need help solving the filenaming problem for hourly rotation.
That’s what somehow worries me, because I’ve never seen something like it. But maybe this has to do with the hourly instead of daily rotation - does this happen for the first rotate on a day, too? If not, I believe to have an explanation for the behaviour:
first a day: logrotate does its thing, successfully.
any other rotate that day: the compress is tried, but: There already is a file of that name (taudit-yymmdd.bz2), so the compress aborts with an error. logrotate notices and aborts, too: The original message file therefor is left as-is.
Maybe the easier way to handle this would be to move the (freshly created) compressed log file in the post-rotate action:
That’s not going to help. I’ve found a way to solve the problem but it created another problem. I’ve got like this long string of problem and each solution to one problem creates a new problem.
To have the information available to work with to solve my file-naming problem, which in turn will fix my log rotation problem, I now need to solve the following problem. I need to get the current time (the current time at the time my shell scrpt runs) stored as an environment variable.
Now, I’ve found that doing the following gets me the info I need:
hwclock | awk ‘{print $5}’
But that just prints the info to the screen, not doing me much good. Instead, I need the result of the above to get stored to a global environment variable called CURRTIME. How can I get this done. If you know, please provide working shell script code.
[QUOTE=ab;4167]-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
It also says %s is allowed, which would be fine since it increments
every second.
Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
Actually I’ve found a better way to get the info I need but I still need a way to export that info to a global environment variable called CURRTIME. This way works better for me to get the time info for what I need to use it for (mainly renaming the logfiles:
On Mon, 23 Apr 2012 21:04:01 +0000, dwoeltje wrote:
[color=blue]
hwclock | awk ‘{print $5}’
But that just prints the info to the screen, not doing me much good.
Instead, I need the result of the above to get stored to a global
environment variable called CURRTIME. How can I get this done. If you
know, please provide working shell script code.[/color]
On Mon, 23 Apr 2012 21:24:01 +0000, dwoeltje wrote:
[color=blue]
Actually I’ve found a better way to get the info I need but I still need
a way to export that info to a global environment variable called
CURRTIME. This way works better for me to get the time info for what I
need to use it for (mainly renaming the logfiles:
date --rfc-3339=ns | awk ‘{print $1"_"$2}’[/color]
Same solution as previous post, just replace the command.
Yea, I finally got that on my own. I gave up on using logrotate and I created a shell script that I put into the /etc/cron.hourly directory. It seems to be running exactly as I need it to. My only problem now is what to do to keep from running out of disk space.
I’m going to need another shell script that I can put in the /etc/cron.monthly directory that will search all the subdirectories beneath the /var/log/audit directory, check the single file that is in each one of those directories to see if it is older than 60 days and, if so, delete the file and the folder it was in. Problem is I won’t know the directory name or the filename in advance. When they both get created (the directory and the file in it) they can a somewhat random name (because the name includes in it the date and time that was the current date and time when my CURRTIME environment variable got “exported”. I’m not sure how I’m going to do this. But I figure I have a couple of months to figure it out. Below is the code in my shell script for rotating the logfiles:
#! /bin/sh
/etc/cron.hourly/taudit-reinit
BEGIN SCRIPT INFO
Description: Re-initialize the fwaudit.log file
END SCRIPT INFO
CURRTIME=date --rfc-3339=ns | awk '{print $1"_"$2}'
export CURRTIME
mkdir /var/log/audit/$CURRTIME
service syslog stop
mv /var/log/audit/taudit.log /var/log/audit/$CURRTIME/taudit2.log
service syslog start
cd /var/log/audit/$CURRTIME
gzip /var/log/audit/$CURRTIME/taudit2.log
mv /var/log/audit/$CURRTIME/taudit2.log.gz /var/log/audit/$CURRTIME/taudit2.log.$CURRTIME.gz
