Simple shared logging
This month we show you how to write a simple logging script that logs messages into a single directory and then collect them into a master log. Once you've got that down, we'll show you how to use the
crondaemon to automatically run this process on a regular basis. (1,200 words)
Sharing a file in shell script is fairly complex, but there is a simpler method of implementing shared logging. The technique involves using a resource that is automatically shared, a directory. The basic plan for sharing would be to write small log files into a logging directory, and then, every so often, collect up the log files and attach them to a larger log.
A simple version of this is illustrated in Listing 1. At line 6, logging directory is chosen by absolute path name. In other words it shouldn't be something like:
$HOME is relative to the user and process
running the program.
The command at line 7 creates a log file name composed of the log directory, date, and time as a string of digits ( `date +%Y%m%d%H%M%S` ) and the process ID of the current process (.$$).
Lines 8 and 9 create open and closed version of the log file name by adding .x and .l extensions.
The actual logging takes place at line 10. The information that is
logged is the date and everything that was passed on the command
logit.sh One two three
would result in a line of logged information that would look like:
Wed Nov 19 16:50:57 PST 1997 One two three
logit.sh -- A simple logging script
Line numbers are included for explanation.
1 #! /bin/sh 2 # logit.sh 3 # log messages to a logging directory by using 4 # a time and job stamp for the file name. 5 6 LOGDIR=/joblogs ; export LOGDIR 7 LOGFILE=$LOGDIR/`date +%Y%m%d%H%M%S`.$$ ; export LOGFILE 8 OPENLOG=$LOGFILE.x ; export OPENLOG 9 CLOSEDLOG=$LOGFILE.l ; export CLOSEDLOG 10 echo `date` "$*" >> $OPENLOG 11 mv $OPENLOG $CLOSEDLOG
One question that immediately pops up is why go through the process of renaming the log files? It is possible to leave all the little log files sitting in the logging directory, but it is much tidier to pick them all up and attach them into one log file. A single log file is easier to read and search.
The problem with grabbing the little log files and collating them into one larger file is that you don't want to try to collate a file that is currently receiving logging information. So a file with a .x extension is a file that is open and should be left alone. A file with a .l extension is finished with, and can be collected into the larger logging file.
The process of collecting the log files into a single file is simple enough on the surface, as in Listing 2. All of the .l files in the log directory are appended to the master log, and the individual files are deleted.
Listing 2: Collecting the log files in a master log
1 #! /bin/sh 2 # bldmaster.sh 3 # collects up all .l files in the logging 4 # directory and appends them to the master log. 5 6 LOGDIR=/joblogs ; export LOGDIR 7 MASTERLOG=master.log ; export MASTERLOG 8 MASTERPATH=$LOGDIR/$MASTERLOG ; export $MASTERPATH 9 10 for name in $LOGDIR/*.l 11 do 12 cat $name >> $MASTERPATH 13 rm -f $name 14 done
The question that arises from listing 2 is, who does all this collecting? If any or all processes could do it, we would just have the same sharing problem all over again as multiple tasks tried to add data to the master log.
So we give the problem to the
cron daemon. The
cron daemon runs in the background on Unix systems. It
is usually started when the system is first booted up. Its job is to
launch tasks that have to be executed at regular intervals. The
types of tasks that you will find a
cron daemon performing
include nightly backups, weekly cleanups of temporary files, and
logging of system messages for the system administrator.
Cron is designed to allow tasks to run as often as
once a minute or as little as once a year.
cron daemon reads one or more files containing
entries that are made up of five columns specifying the times when a
process should run, one column naming the program or script with its
command line arguments, and an optional comment field.
cron table for nightly backups might look
like Listing 3. The weekday varies from 0 for Sunday through 6 for
Saturday. The backup time is 11:20 p.m., except for Sunday and
Saturday when it runs at 7:20 p.m.. The day and month fields are
marked with an asterisk meaning any legal value. Or in this case, any
month or any day.
Listing 3: A sample
#Min Hr Day Month WeekDay Job #------------------------------------------------------------- 20 19 * * 0 /prgms/backup/sunbu.sh #Sunday Backup 20 23 * * 1 /prgms/backup/monbu.sh #Monday 20 23 * * 2 /prgms/backup/tuebu.sh #Tuesday 20 23 * * 3 /prgms/backup/wedbu.sh #Wednesday 20 23 * * 4 /prgms/backup/thubu.sh #Thursday 20 23 * * 5 /prgms/backup/fribu.sh #Friday 20 23 * * 6 /prgms/backup/satbu.sh #Saturday
You can specify multiple values for a column by separating the
values with commas. In Listing 4 the
has been added to run every 15 minutes. Note that the hour, day,
month, and weekday are filled in with asterisks, so the process runs
every 15 minutes of every day of the week.
Listing 4: A sample
cron file with the master logging process.
#Min Hr Day Month WeekDay Job #------------------------------------------------------------- 20 19 * * 0 /prgms/backup/sunbu.sh #Sunday Backup 20 23 * * 1 /prgms/backup/monbu.sh #Monday 20 23 * * 2 /prgms/backup/tuebu.sh #Tuesday 20 23 * * 3 /prgms/backup/wedbu.sh #Wednesday 20 23 * * 4 /prgms/backup/thubu.sh #Thursday 20 23 * * 5 /prgms/backup/fribu.sh #Friday 20 23 * * 6 /prgms/backup/satbu.sh #Saturday 0,15,30,45 * * * * /prgms/bldmaster.sh #Collate logs
cron process keeps a list of
jobs for each user legally allowed to create
If you are root, you will be able to create
cron jobs by editing the root
table. Proceed with caution as you type:
cron table will be opened by the vi editor.
If you are not root, but you do have permission to create
cron jobs, typing
crontab -e will open
cron table for editing. If you are not
cron, this will probably be a new file.
If you are just learning about or experimenting with
cron, be sure to back up your
tables. These are probably in
/var/spool/cron/crontabs, but see your manual for
the location of
If you do not have
cron rights, you must create
bldmaster.sh process, and then get your system
administrator to install it as a
About the author
Mo Budlong is president of King Computer Services Inc. and has been involved in Unix development on Sun and other platforms for over 15 years. King Computer Services Inc. specializes in Unix and Client/Server consulting and training and currently publishes the COBOL Just In Time Course, a crash COBOL course to train staff for the Year 2000 problem. Reach Mo at firstname.lastname@example.org.
If you have technical problems with this magazine, contact email@example.com