SysAdmin

The shell game

Your shell environment is full of handy tools and shortcuts.

The shell is something used so often that it gets taken for granted by advanced users; that familiarity frustrates and confuses the beginner. The innards of a shell may seem as interesting as the physics that make your desk chair feel comfortable. The value in this addition to your brain cache is a stronger ability to solve user problems.

What the shell?
Shells have proliferated to meet the demands of Unix users. The original, no-frills Bourne shell gives you a functional command line interpreter and access to multitasking through background processes. Add in C language syntax, job control, a history mechanism, and filename completion, and you get the C shell. System V brings the Korn shell, with Bourne shell-like built-in operators, more elaborate history and aliasing mechanisms, and a line-editing feature that speaks both vi and emacs.

All shells employ the notion of a login shell, nominally the shell you entered courtesy of /bin/login. But there's no simple way to determine if your shell was the first spawned, so the first character of the command name is used as an indicator. If it's a minus sign, the shell is assumed to be a login shell. Login makes the distinction by changing /bin/csh to -csh in the command line passed to exec(). You'll see these tweaked command names in the output of ps.

Bourne and Korn shells get going by reading the global /etc/profile, followed by the user's .profile. Korn shells complete setup by looking for the file named by the variable ENV, defaulting to .env, executing it if found. All Bourne and Korn shell initialization occurs for login shells only. Create common user environments in /etc/profile, but be sure all of your users employ the same shell to avoid unpleasant surprises.

The C shell consumes the user's .cshrc followed by the .login file in a login shell. Confusion can result after using .login to unintentionally re-initialize variables correctly set in .cshrc. If the user doesn't have a .login file, a common /etc/.login file is read. Your .cshrc is always read, even for non-login shells, which contributes to the start-up time of the C shell.

Getting to /bin/yes
Every line you type gets passed to the shell for inspection, substitution, and execution. Before invoking the command you so dutifully typed, the shell has many duties of its own:

• Redirecting file descriptors for stdin, stdout, and stderr. The Bourne and Korn shells let you redirect any file descriptor by number, using the 2>& file syntax.
• Alias expansion, substituting typed command names with long-hand aliases or sequences of built-in commands.
• Substitution of variables and filename wildcard characters. Typing rm * makes work for the shell, expanding the wildcard to a list of every file in the current directory. Like redirection, this feature is powerful in its simplicity and consistency: The shell builds up a complete command line so that individual commands don't need to worry about handling wildcards. Not all shells are this hardworking. Type erase *.* in a DOS shell, and erase has to do the work of finding all of the files.
• Locating the executable, searching the directories in the PATH variable. The Unixexec() call assumes you know where the image lives, placing the burden on the caller to find it. All shells build a hash table of commands and use it for quick location in lieu of searching every directory in the PATH variable. Change your path and the hash table is rebuilt, but if you add to an existing directory, you'll need to force a rebuild to find the new tool. The C shell function is rehash. When users can't find a recently installed tool, update the hash tables in their shells before looking deeper.
• Building the argument list, using the characters in the IFS (internal field separator) variable as demarcation points. Although the default IFS set includes a tab, space, and new line, you can add to the collection to ease shell programming. The Solaris start-up routines change the IFS to include a dot so that filenames like hostname.le0 are treated as two arguments (hostname le0) during subsequent parsing.

The last character in a command line is an ampersand (&) if you want the job to run in the background. The Bourne shell lets you start background jobs, but gives you no way to move between them once started. The C and Korn shells implement job control so you can suspend, resume, and move jobs between foreground and background status.

Line discipline
The interaction between keyboard and shell subsumes the tty driver and its configuration (known as a line discipline), signal handling, interaction with the window system, and command line editing. Normally, each character you type at the keyboard is passed to the tty driver, only making it to the shell when you hit Return. Cut and paste between windows is a special case; it looks like input typed at the keyboard, but it's the window system and not your fingers that supplies the bits. Character echo is a function of the tty driver, as are higher-level functions like character erase, word erase, and line kill. To wipe out an entire input line in a fit of frustration, use control-U (line kill). Binding of line discipline functions to control keys is handled by thestty command, which also lets you remap the suspend, interrupt (SIGINTR), and quit (SIGQUIT) signal-generation keys.

Goodies like stty, echo, or read commands belong in the .login file. They'll have bizarre effects on the rcp, rsh, and rdump (rmt) commands if they appear in your .cshrc file. When the Berkeley r commands are used, they spawn an instance of/bin/csh on the remote machine, which executes your .cshrc file. Any unexpected output from the shell start-up confuses the local utility, resulting in some very cryptic Unix error messages.

History rewritten
The Korn shell (and the tcsh shell) adds another level of input with line editing. Enable this with a set -o vi or set -o emacs in your .profile. You start in vi input mode, so what you type becomes shell input. Hit Escape, and you're in command-editing mode where most of your favorite editor sequences work.

The C shell offers filename completion. When enabled (set filec in your .cshrc file), pressing Escape expands a unique prefix into a full filename.

The C shell history mechanism is another re-execution mechanism, with sed-like commands replacing the in-line editing of the Korn shell. The history lives in the shell's memory history; depth is set by the variable history. If not set, no history is maintained. savehist sets how many commands are stored in $HOME/.history upon logout, used to seed the history stack on your next login.

Korn shell history is entirely file-based. Recently executed commands go into $HOME/.sh_history, or the filename specified by $HISTFILE. When multiple shells share one history file, their commands are intermixed. Separate the history for each shell, putting a line of the form HISTFILE=$HOME/.sh_history$$ in your .profile. Be sure to clean up old history files before logging out.

The ksh built-in fc calls an editor (named in the FCEDIT variable) to modify commands from the history file. Giving - as an editor name reduces the edit-execute cycle to a replace-and-execute sequence similar to the C shell. For example, fc -e - lotus in ksh is equivalent to !lotus in csh; fc -e - is usually aliased to a single character, such as r.

Cruiser's compendium
There are numerous shell environment issues worth mention. Here are some additional pointers and caveats:

• Command names are matched first against keywords, then in the list of aliases, and then in the shell's hash table. Don't let users create executables called test, or/usr/ucb/test might run before the test image in the current directory.
• Watch dynamic shell script execution using the -x option to any shell, for example, sh -x newscript. You'll see variable substitution in progress, with each command preceded by a plus sign.
• If you must run a verbose command inside your .cshrc or .profile, redirect the standard output and standard error messages to /dev/null to do Berkeley r-command damage control:
  

  noisy >& /dev/null # csh
noisy > /dev/null 2>&1 # sh, ksh


  In the Bourne and Korn versions of the line, the 2>&1 construct means "duplicate file descriptor 2 from file descriptor 1," or send stderr to stdout. Since stdout was already redirected to /dev/null, this sends all output into the bit bucket. Redirection arguments are order-sensitive; if you put the stderr and stdout redirection first, stderr will go to whatever stdout was, and only stdout will go to /dev/null.

• Don't call other shell scripts from your .cshrc file and expect them to affect your environment. Read a common user profile "in line" using the command:
  

  source /usr/local/config/common.csh # csh
. /usr/local/config/common.sh # sh, ksh


  Executing the script as a command applies the changes to a subshell, which promptly exits at the end of the script, leaving your login shell none the wiser.

• If you want to maintain a user's ID on a machine (for correct expansion of ~user and ls output, for example) but don't want to allow login access, edit the password file and replace the shell field for that user with /bin/true (which exits immediately). This technique is easier and more firm than policing /etc/hosts.equiv and individual users' .rhosts files for hundreds of machines and users.
• Users who employ shells other than the Big Three will require entries in the /etc/shells file for proper ftp operation. If users get Permission denied failures when logging into a remote machine with ftp, check that the user's shell is ftp-enabled on the target machine.
• Even if you are a blue-blood C shell user, you'll have to cope with the Bourne shell for root under System V Release 4 and derivatives, including Solaris 2.x. You can fire up a C or Korn shell as soon as you become root, but root's shell must be /bin/sh since all of the start-up scripts are written in the Bourne shell.
• Keep your path short. If you list automounted directories in your path, each will be mounted during shell start-up as the executable hash table is built. In addition to slowing down shell initialization and causing an automounter storm, a long path leaves you exposed to NFS server hangs. If one of the NFS servers crashes or is down while you're starting a shell, the shell will hang, waiting for the server to come back to life.

We'll cover variable substitution, evaluation, execution, and other mechanisms that make the shell intriguing and powerful next month. Explore your shell environment; you'll find it full of shortcuts and tools that make you bolder with sleights of hand and keyboard.

Check out the sidebars Big shells, no perl and Models and options.

About the author
Hal Stern is an area systems engineer with Sun Microsystems in the Northeast, where he specializes in networking, performance tuning, and kernel hacking. He can be reached at hal.stern@sunworld.com.

You can buy Hal Stern's Managing NFS and NIS at Amazon.com Books.

(A list of Hal Stern's Sysadmin columns in SunWorld Online.)

Big shells, no perl

• If you're intrigued by the Korn shell but fear SVR4, check out pdksh. It's not completely ksh-compatible, but it's a good start. Look in /pub/src/shells.
• GNU brings you bash, the Bourne Again Shell. Based on the Bourne shell syntax, it adds command line editing, job control, history, and various C shell-like wildcard expansions. Version 1.12 is available in /pub/src/gnu.
• Mush, the Mail User's Shell, was written by Dan Heller and combines C shell syntax with a mail user interface. It's in /pub/src/mail.
• Former VMS users who enjoy cruising through command history with arrow keys will like tcsh, an extended version of the C shell. Other features include command completion and emacs-style line editing. Look in /pub/src/shells.

Models and options

                 Bourne         C                       Korn 
----------------------------------------------------------------------------
Job control      no             yes                     yes
History          no             yes, login "memory" in  yes, with editing per
                                $HOME/.history          shell in $HISTFILE,  
                                                        can combine shell 
                                                        histories in one file
Global setup     /etc/profile   /etc/.login if none     /etc/profile
                                exists for user
Line editing     no             filename completion,    vi or emacs style
                                more in tcsh
File descriptor  by number      stdin, stdout, or       by number and via
redirection                     stdout/stderr           process substitution












If you have problems with this magazine, contact webmaster@sunworld.com


URL: http://www.sunworld.com/asm-03-1995/asm-03-sysadmin.html.

Last updated 1 March 1995.