CS300, Scripts and Utilities

Bourne shell, /bin/sh ... part 2



Here is the Debian reference for shell programming.


Like csh, the Bourne Shell waits for commands to finish before continuing on. To execute a command and not wait until it finishes (run it in the background), append an ampersand to the command: e.g.

% xterm &
This executes the xterm program, but lets you continue without waiting for the xterm program to finish.

Combining commands: semi-colon and ()

Besides pipes, there are two other ways that the shell lets you combine commands. First is the semi-colon -- this allows you to combine multiple commands on one line. Thus lshome2 is really the same as lshome , except all three commands are on one line.

In and of itself, this is not very exciting. However, it becomes more powerful when you combine it with parentheses. With parentheses, you combine multiple commands and execute them in a sub-shell. For example, suppose you would like to time how long it takes to ping a machine at princeton. One way to do this is to use the time command. However, a more primitive way is to simply call date before and after the command. If you combine them all on one line with semi-colons, then you get a fairly accurate timing:

UNIX> sh
$ date ; /usr/sbin/ping www.cs.princeton.edu ; date
Fri Jun  6 08:56:44 EDT 2001
www.cs.princeton.edu is alive
Fri Jun  6 08:56:46 EDT 2001
$
Now, suppose you'd like the output of the three commands to go to a file. Of course, one way is to redirect each command to the output file:
$ date > out ; /usr/sbin/ping www.cs.princeton.edu >> out ; date >> out
Another way is to bundle up the three commands inside parentheses, and redirect the output of the composite command to the file:
$ ( date ; /usr/sbin/ping www.cs.princeton.edu ; date ) > out
You can also use parentheses to put a composite command into the background:
$ ( date ; /usr/sbin/ping www.cs.princeton.edu ; date ) > out &


echo

echo is a shell command that simply prints its arguments on standard output. For example:
$ echo Hello
Hello
$ echo Hello    World 
Hello World
$ echo

$
Note that echo separates multiple arguments by a single space. I could keep that additional space between the words by quoting the arguments, essentially making it all one argument
$echo "Hello    World"
Hello    World 

Filename expansion: * and ?

You can type any string into the shell, but certain characters are special (like '>', '<', '|', etc). You should already know about the asterisk (star) -- this expands as a wildcard to match any filename: e.g. ``echo *'' echos all filenames in the current directory (excepting those that start with '.'). ``echo lshome*'' echos both ``lshome'' files.
$ echo *
bq1 bq2 count doubleprintarg1 f1 f2 ifcat input1 lecture.html logo.gif lshome lshome2 outline printarg1 setexample simple sortword specialvar testfor testfor2 testforone tfo2 whatsmyname wmn2
$ echo lshome*
lshome lshome2

The question mark is a wild card that will match any one character. Thus, ``echo ??'' will print out all filenames in the current directory that are composed of two characters, and ``echo lshome?'' will print out ``lshome2'', since the question mark must match one character:
$ echo ??
f1 f2
$ echo lshome?
lshome2

Single and double quotes

Quotes allow you to:
  1. Use special characters in strings
  2. Bundle up multiple words into one argument
Single quotes allow you to use most any character in a string. For example, you can use *, ?, (, ), >, <, |, ", $, & and space in single quotes without having the shell do anything special to them:
$ echo Do you have $100 (so I can borrow it?)
syntax error: `(' unexpected
$ echo 'Do you have $100 (so I can borrow it?)'
Do you have $100 (so I can borrow it?)
$ echo 'Hey   'ya'!'
Hey   ya!
$ echo 'Hey   ' ya'!'
Hey    ya!
$ echo 'Hey   '               ya'!'
Hey    ya!
Double quotes are less powerful. They work like single quotes except $ gets expanded (see below).
$ echo "$HOME"; echo '$HOME'
You can build strings simply by concatenating them as above. To get a single quote in a string, you need to use double quotes, and to get a double quote in a string, you need to use single quotes. For example, to get the string "'", you do the following:
$ echo '"'"'"'"'
"'"
$ echo 'She said "This is Plank'"'s course!!"'"'
She said "This is Plank's course!!"

Differences between csh and the Bourne shell

There are many differences between the Bourne shell and csh. Some major ones are that the Bourne shell has no command aliasing, no history, no '~' expansion, no 'setenv', and slightly different syntax for redirection. For example, there is no '2>&1' in csh. Instead, if you want to standard output and standard error to go to the same place, you use '>&' There are other major differences in expression syntax, etc. Since I almost never program with csh, I don't know all the differences. However, in general, you can't use csh on a Bourne shell script.

Fortunately, the use of single and double quotes is nearly identical in both the Bourne shell and csh.


Environment variables

Environment variables are a simple associative matching between names and strings. The Bourne shell will inherit the environment variables in its calling environment (i.e. any setenv's that you have done in the csh), and it lets you set your own.

Environment variables are expanded by using the dollar sign. For example, your home directory and user name are always in the environment variables HOME and USER respectively:

$ echo $HOME
/homes/username
$ echo $USER
username
$
As stated previously, environment variables are expanded in double quotes, but are not in single quotes. You can use quotes to build strings out of environment variables.
$ echo "$HOME"
/homes/username
$ echo '$HOME'
$HOME
$ echo $HOME$USER
/homes/usernameusername
$ echo $HOME $USER
/bay/homes/username username
$ echo "$USER's home directory is $HOME"
username's home directory is /homes/username 
$
When you're running a shell script, you can get at the command line arguments using $1, $2, $3, up to $9. For example, printarg1 prints out the first command line argument:
$ printarg1 1
1
$ printarg1 Hello World 
Hello
$ printarg1 "Hello World"
Hello World
$
When programming with the Bourne shell, you often have to be careful to use double quotes whenever you may get a space in a string. For example, look at doubleprintarg1. Make sure you understand why the output of doubleprintarg1 below is as it is:
$ doubleprintarg1 "Hello World"
Hello
Hello World
$
To set an environment variable, you do ``var=string''. Note that there should be no space between the equals sign and the var and string. In setting the environment variable, the only way that you can use spaces in the string is to enclose it in quotes. Examples:
$ a=Hello
$ b=World
$ echo $a $b
Hello World
$ c="Hello World"
$ echo $c
Hello World
$ d=$a $b
World: not found
$ d="$a $b"
$ echo $d
Hello World
$ d=$d$d
$ echo $d
Hello WorldHello World
$

Special environment variables: $#, $$, $! $*

There are a few special environment variables: Look at specialvar, and make sure you understand the output when executed with the following:
$ specialvar
Number of arguments: 0
Process id: 4699
Arguments: 
Forked process pid: 4700
$ specialvar a1 a2 a3
Number of arguments: 3
Process id: 4701
Arguments: a1 a2 a3
Forked process pid: 4702
$ specialvar GIVE     HIM    SIX
Number of arguments: 3
Process id: 4705
Arguments: GIVE HIM SIX
Forked process pid: 4706
$

For those of you who are not familiar with UNIX devices, /dev/null may seem very odd. But the system treats it as a file. It has some special properties that make it very handy. It never has anything in it, always returning an end-of-file immediately when it is read. It also 'absorbs' everything that is written to it. Which makes it very handy if you don't want output written to a file or the screen, like perhaps error message.


If statements

The syntax of if statements is:
if bool
  then
     statements
fi
Usually, you use a semicolon and put the then on the same line:
if bool ; then
     statements
fi
You can have an else clause, and any number of elif clauses. Now, what is the boolean statement? Unix processes return a number to their caller (for those who have taken CS360, this is exit value returned by the wait() system call). The boolean statement is a Unix command, and if it returns zero, then the then part of the clause is executed. Otherwise, the next elif or else clause is executed.

For example, cat returns 0 if it runs successfully, and 1 if it encounters an error. Look at ifcat:

$ cat ifcat
#!/bin/sh

if cat $1 > /dev/null 2>&1 ; then 
  echo "cat $1 worked just fine"
else
  echo "cat $1 returned with an error"
fi
$
This executes cat on the argument, and uses the exit value of cat to report whether it was successful. Try it out:
$ ifcat f1
cat f1 worked just fine
$ ifcat /usr/dict/words             
cat /usr/dict/words worked just fine
$ ifcat no-such-file
cat no-such-file returned with an error
$ 

The Test Program

There is a program called test whose purpose is to evaluate boolean functions. Do man test to learn the complete syntax. I'll go over a few: As an example, whatsmyname is a simple shell script that has you guess my name on the command line:
$ whatsmyname Jim
Right!
$ whatsmyname James
Right, although I prefer to be called Jim
$ whatsmyname Peyton
Nope
$ whatsmyname Frank
"Frank Plank" -- Are you kidding?!?!?!?!
$
To improve readability, the Bourne shell lets you enclose your arguments to test in square brackets. Then if statements look much better. wmn2 is pretty much just like whatsmyname, except that it uses the square brackets, and handles some conditions better than whatsmyname:
$ whatsmyname
whatsmyname: test: argument expected
$ wmn2
usage: wmn2 name
$ wmn2 Jim
Right!
$ whatsmyname Jim Plank
Right!
$ wmn2 Jim Plank
usage: wmn2 name
$ whatsmyname "Jim Plank"
whatsmyname: test: unknown operator Plank
$ wmn2 "Jim Plank"
Nope
$ 

While

While's syntax is similar to if's:
while bool ; do
     statements
done

Shift

Shift is a simple command that shifts the command line arguments by one. The $# and $* variables are changed as well. For example, testforone uses shift in a while loop to test if any of its command line arguments equal one. Note that the numerical value of a string is its atoi() value. This is an integer, not a floating point number.
$ testforone 
$ testforone 3 2 1 1.0 1.5 "1 Jim" "Jim Plank"
No:  3 does not equal 1
No:  2 does not equal 1
Yes: 1 equals 1
Yes: 1.0 equals 1
Yes: 1.5 equals 1
Yes: 1 Jim equals 1
No:  Jim Plank does not equal 1
$

Set

The set command lets you set the command line arguments to something else. This can be very convenient. As an example, look at setexample. It sets the command line arguments and then prints them out one by one with a while/shift loop. Note the use of quotes in "Rocky Top".
$ setexample
Once I Had a Girl on Rocky Top
Once
I
Had
a
Girl
on
Rocky Top
$ setexample XXX OOO
Once I Had a Girl on Rocky Top
Once
I
Had
a
Girl
on
Rocky Top
$ 

For

You can do for loops in one of two ways:
for var do
  statements
done
or
for var in strings ; do
  statements
done
The former way loops through all command line arguments, each time setting the var to be the argument. The latter way loops through each string, each time setting the var to be the string.

For example, tfo2 uses the first kind of for loop to implement a program equivalent to the testforone program:

$ tfo2
$ tfo2 3 2 1 1.0 1.5 "1 Jim" "Jim Plank"
No:  3 does not equal 1
No:  2 does not equal 1
Yes: 1 equals 1
Yes: 1.0 equals 1
Yes: 1.5 equals 1
Yes: 1 Jim equals 1
No:  Jim Plank does not equal 1
$ 
For example, testfor shows a very simple example of the second kind of for loop:
$ testfor
Once
I
had
a
girl
on
Rocky Top
$ 

Line Continuations

You can end a line with a backslash to continue it to the next line, as in testfor2.

Backquotes

Backquotes are very important in the Bourne Shell. They execute the command in the backquotes, and then treat the output like a list of strings. If the output is on multiple lines, it is treated as one big list of strings with no newlines. For example, bq1 takes a file as its argument and prints out the content of the file all on one line:
$ bq1 f1
This is f1
$ bq1 bq1
#!/bin/sh if [ $# -ne 1 -o ! -f "$1" ]; then echo "usage: bq1 filename" >& 2 exit 1 fi b=`cat "$1"` echo $b
$ 
And bq2 takes a file as its argument and prints out each word on its own line:
$ bq2 f1
This
is
f1
$ bq2 input1
Once
I
had
a
girl
on
Rocky
Top
Half
bear
the
other
have
cat
Mean
as
a
snake
but
sweet
as
soda
pop
I
still
think
about
that
You can use parentheses around larger blocks of code to get some nice effects. For example, sortword tweaks bq2 to sort the words in a file. How long would it have taken you to write that in C?
$ sortword input1
Half
I
Mean
Once
Rocky
Top
a
about
as
bear
but
cat
girl
had
have
on
other
pop
snake
soda
still
sweet
that
the
think
$

bc

bc is a simple infix calculator that often gets used to do induction variables in sh. For example, myinc prints out the command line argument plus one using bc:
$ myinc 4
5
$
Note that using bc in loops is pretty slow (for example, the square program that you'll write as part of your lab is much, much slower than a C version would be). That's because each time you call bc and test you're forking off a new process. This is expensive compared to doing everything in a C program. That's why it's best to think of shell scripts as something that you write when efficiency of writing the program is more important than efficiency of the program itself.

expr is another program that lets you do math in shell scripts. It's nicer than bc because it lets you specify the arguments on the command line, and it performs math, logical arithmetic, and string manipulation. However, it does not do floating point arithmetic. Read the man page for more information. For example, einc is just like myinc except it uses expr instead of bc. You'll notice that you cannot increment decimal numbers with einc, but you can with myinc.


Other stuff

You can read about other stuff in the man page for sh. Other things that you may want to know about are: