|
Simple shared logging
Using the
|
This month we show you how to write a simple logging script that logs messages into a single directory and then collect them into a master log. Once you've got that down, we'll show you
how to use the cron
daemon to automatically run this process on a regular basis. (1,200 words)
Mail this article to a friend |
Sharing a file in shell script is fairly complex, but there is a simpler method of implementing shared logging. The technique involves using a resource that is automatically shared, a directory. The basic plan for sharing would be to write small log files into a logging directory, and then, every so often, collect up the log files and attach them to a larger log.
A simple version of this is illustrated in Listing 1. At line 6, logging directory is chosen by absolute path name. In other words it shouldn't be something like:
LOGDIR=$HOME/logs
because $HOME
is relative to the user and process
running the program.
The command at line 7 creates a log file name composed of the log directory, date, and time as a string of digits ( `date +%Y%m%d%H%M%S` ) and the process ID of the current process (.$$).
Lines 8 and 9 create open and closed version of the log file name by adding .x and .l extensions.
The actual logging takes place at line 10. The information that is
logged is the date and everything that was passed on the command
line to logit.sh
.
The command:
logit.sh One two three
would result in a line of logged information that would look like:
Wed Nov 19 16:50:57 PST 1997 One two three
|
|
|
|
Listing 1: logit.sh
-- A simple logging script
Line numbers are included for explanation.
1 #! /bin/sh 2 # logit.sh 3 # log messages to a logging directory by using 4 # a time and job stamp for the file name. 5 6 LOGDIR=/joblogs ; export LOGDIR 7 LOGFILE=$LOGDIR/`date +%Y%m%d%H%M%S`.$$ ; export LOGFILE 8 OPENLOG=$LOGFILE.x ; export OPENLOG 9 CLOSEDLOG=$LOGFILE.l ; export CLOSEDLOG 10 echo `date` "$*" >> $OPENLOG 11 mv $OPENLOG $CLOSEDLOG
One question that immediately pops up is why go through the process of renaming the log files? It is possible to leave all the little log files sitting in the logging directory, but it is much tidier to pick them all up and attach them into one log file. A single log file is easier to read and search.
The problem with grabbing the little log files and collating them into one larger file is that you don't want to try to collate a file that is currently receiving logging information. So a file with a .x extension is a file that is open and should be left alone. A file with a .l extension is finished with, and can be collected into the larger logging file.
The process of collecting the log files into a single file is simple enough on the surface, as in Listing 2. All of the .l files in the log directory are appended to the master log, and the individual files are deleted.
Listing 2: Collecting the log files in a master log
1 #! /bin/sh 2 # bldmaster.sh 3 # collects up all .l files in the logging 4 # directory and appends them to the master log. 5 6 LOGDIR=/joblogs ; export LOGDIR 7 MASTERLOG=master.log ; export MASTERLOG 8 MASTERPATH=$LOGDIR/$MASTERLOG ; export $MASTERPATH 9 10 for name in $LOGDIR/*.l 11 do 12 cat $name >> $MASTERPATH 13 rm -f $name 14 done
The question that arises from listing 2 is, who does all this collecting? If any or all processes could do it, we would just have the same sharing problem all over again as multiple tasks tried to add data to the master log.
So we give the problem to the cron
daemon. The
cron
daemon runs in the background on Unix systems. It
is usually started when the system is first booted up. Its job is to
launch tasks that have to be executed at regular intervals. The
types of tasks that you will find a cron
daemon performing
include nightly backups, weekly cleanups of temporary files, and
logging of system messages for the system administrator.
Cron
is designed to allow tasks to run as often as
once a minute or as little as once a year.
The cron
daemon reads one or more files containing
entries that are made up of five columns specifying the times when a
process should run, one column naming the program or script with its
command line arguments, and an optional comment field.
A sample cron
table for nightly backups might look
like Listing 3. The weekday varies from 0 for Sunday through 6 for
Saturday. The backup time is 11:20 p.m., except for Sunday and
Saturday when it runs at 7:20 p.m.. The day and month fields are
marked with an asterisk meaning any legal value. Or in this case, any
month or any day.
Listing 3: A sample cron
file
#Min Hr Day Month WeekDay Job #------------------------------------------------------------- 20 19 * * 0 /prgms/backup/sunbu.sh #Sunday Backup 20 23 * * 1 /prgms/backup/monbu.sh #Monday 20 23 * * 2 /prgms/backup/tuebu.sh #Tuesday 20 23 * * 3 /prgms/backup/wedbu.sh #Wednesday 20 23 * * 4 /prgms/backup/thubu.sh #Thursday 20 23 * * 5 /prgms/backup/fribu.sh #Friday 20 23 * * 6 /prgms/backup/satbu.sh #Saturday
You can specify multiple values for a column by separating the
values with commas. In Listing 4 the bldmaster.sh
process
has been added to run every 15 minutes. Note that the hour, day,
month, and weekday are filled in with asterisks, so the process runs
every 15 minutes of every day of the week.
Listing 4: A sample cron
file with the master logging process.
#Min Hr Day Month WeekDay Job #------------------------------------------------------------- 20 19 * * 0 /prgms/backup/sunbu.sh #Sunday Backup 20 23 * * 1 /prgms/backup/monbu.sh #Monday 20 23 * * 2 /prgms/backup/tuebu.sh #Tuesday 20 23 * * 3 /prgms/backup/wedbu.sh #Wednesday 20 23 * * 4 /prgms/backup/thubu.sh #Thursday 20 23 * * 5 /prgms/backup/fribu.sh #Friday 20 23 * * 6 /prgms/backup/satbu.sh #Saturday 0,15,30,45 * * * * /prgms/bldmaster.sh #Collate logs
The cron
process keeps a list of cron
jobs for each user legally allowed to create cron
jobs.
If you are root, you will be able to create
cron
jobs by editing the root cron
table. Proceed with caution as you type:
crontab -e
Your root cron
table will be opened by the vi editor.
If you are not root, but you do have permission to create
cron
jobs, typing crontab -e
will open
your personal cron
table for editing. If you are not
familiar with cron
, this will probably be a new file.
If you are just learning about or experimenting with
cron
, be sure to back up your cron
tables. These are probably in
/var/spool/cron/crontabs
, but see your manual for
the location of cron
tables.
If you do not have cron
rights, you must create
your bldmaster.sh
process, and then get your system
administrator to install it as a cron
job.
|
Resources
About the author
Mo Budlong is president of King Computer Services Inc. and has been
involved in Unix development on Sun and other platforms for over 15
years. King Computer Services Inc. specializes in Unix and Client/Server
consulting and training and currently publishes the COBOL Just In Time Course,
a crash COBOL course to train staff for the Year 2000 problem.
Reach Mo at mo.budlong@sunworld.com.
If you have technical problems with this magazine, contact webmaster@sunworld.com
URL: http://www.sunworld.com/swol-12-1997/swol-12-unix101.html
Last modified: