CS300, Scripts and Utilities


Bourne shell, /bin/sh ... part 1

This is the first Class session of CS300 Scripts and Utilities. Today I hope to actually start discussing /bin/sh, the Bourne shell. First though a little accounting. Look at the syllabus which is very general. Then I need to make sure that all of you have accounts in the CS departments, so that you can login on these machines.

For those of you who may need an account, I want to stress that this account does not give you unlimited access nor storage on the Department's computers nor will it continue past this semester unless you are also taking courses which are part of the Computer Science major beyond CS102.

For those of you who already have accounts, please login so we can see if there are any problems.

A few of questions for all of you. If the answer to any of these is 'NO', raise your hand. 1. Do you know basic UNIX file system? 2. Do you understand file permissions and how to change them? (chmod) 3. Do you know how to use some email program under UNIX? (pine,mh,mailx) 4. Do you know how to use some editor under UNIX? (vi,pico,vim,gvim)

man

This utility is used to find and display reference manual pages. This is the UNIX online help system. Unfortunately, it requires that you already have some idea of what is going on before you use it. Which is another reason for this discussion. Another unfortunate thing about man is that the 'names' of many of the pages are duplicated and it is often difficult for the people managing them to be sure that the most current ones and all of those are readily available. Our lab staff is making a serious effort to ensure that the correct pages are on the system but occasionally they miss something. Don't get angry when you can't find something or the page doesn't match what's happening. Simply deal with the problem and send mail to "help" explaining the problem. Please try not to get upset if "help" replies back that the page is the correct one and you are doing something wrong. It may simply mean that you are looking at different pages. Ensure that you let help know what your MANPATH is and in which section and path you are looking. If you cannot resolve the problem with email, talk to the Lab Assistant manning the desk and hopefully you can come to some understanding.

man is invoked by typing the word 'man' at the Unix prompt followed by the name of a command, utility, function or word you hope might be there. If there is a manual page available for that item in the users MANPATH, the manual page is displayed using the users default PAGER (eg. more). We will discuss MANPATH and PAGER later.

Most pages consist of a header which indicates which section of the manual you are in currently, the name of the man-page, a synopsis of the command/utility/function usage and then a description. 'man man' should get you something that looks like this 

User Commands                                              man(1)

NAME
     man - find and display reference manual pages

SYNOPSIS
     man  [ - ]  [ -adFlrt ]  [ -M path ]  [  -T macro-package  ]
     [ -s section ] name ...

     man  [ -M path ] -k keyword ...

     man  [ -M path ] -f file ...

DESCRIPTION
The section is usually displayed in the top right hand corner in parentheses, (1) in this case. This is a broad outline of a man-page. There are so many differences in pages that being any more specific would give you the wrong idea.

There are several different command line options to man. For a listing of all the manual pages for the "item" use 'man -l item' For all the man-pages for a specific name use 'man -a item'. If you don't know the name but think it has to do with a specific topic you can try 'man -k topic'. This is the "keyword" search and may just give you a screen full of useless info but... A reason for using the '-l' option is so that you can chose which page to look at if there are multiple pages for an item. 

Lets try the following

man man
  to read the manual page for the command man.

man -a print
  this may return anywhere from 1 to 4 or perhaps more pages.
  if it returns more than one, you need to figure how to use your
  pager to display them all.  For more just continue to page down.
  If any of you are using less, :n goes to the next 
  file/man-page.

man -l read
  You get several listings.  You might note that some
  are from /usr/man and some from /usr/local/man.  If you look at
  pages from the corresponding sections you will find that they are the
  same.

man -S2 read
  This gets you the manual page on the C programming language command 
  read(), which is man page section 2.  Many, but by no means all, of 
  the C commands are in section 2.  And if you don't specify this section, 
  or -a you will probably get something other than what you want.  Try

man read.

This is all I am going to say about man. Other than to warn you that a common response to a request for more in-depth information will be "read the man-page".

The Shell

So what exactly is the shell?

The UNIX Shell is a UNIX program which gives users access to operating system features for launching programs and managing files. The shell can be used interactively or non-interactively from lists of shell commands stored in files called shell scripts. Some shells provide a set of menus (rather than a command line) from which a user can make selections. We are NOT going to be using that type of shell. Commands entered by the user are passed by the shell to the operating system which executes them. The results are then passed back by the shell and displayed for the user. A shell can be any "easy-to-use" user interface.

I said earlier that I would talk about MANPATH and PAGER and this seems like a reasonable time to do it. How many of you are using csh? How many are using some other shell? How many don't know? Well, for those who have not explicitly requested to have your shell changed, in this department you are probably using csh. If you would like to be sure try 

UNIX> echo $SHELL
If you want to know more about echo, try the man pages. The reason I ask is to try to tailor this next bit if necessary. When you first login the process that handles login does some things besides checking your password and user name. It also finds out which shell you use and starts up a copy of that shell using some default files. Part of this operation sets some environment variables. These variables are used by certain programs to keep track of information and determine how to act in some situations. There are many of these and all shells don't use the same variables but there is a subset that you can pretty much depend on. Here are some of the most common:

These can be checked by typing 

UNIX> echo $VARIABLE_NAME
at the prompt. These are also defined in the dot file .cshrc. When you log in the operating system opens the .cshrc and(or) .profile file and reads your default settings. You can view your default settings by typing 
UNIX> more .cshrc (or)
UNIX> more .profile
For some shells, any or all of these may be lower case. If some program doesn't run properly but you can't see why, check the man-page for that program and see what if any environment variables are used. If it still does not make sense, ask me if it has to do with class or mail "help".

Summary of shell facilities

      
                    Bourne   C    TC   Korn   BASH
________________________________________________________


command history        No   Yes   Yes   Yes    Yes

command alias          No   Yes   Yes   Yes    Yes

shell scripts          Yes  Yes   Yes   Yes    Yes

filename completion    No   Yes   Yes   Yes    Yes

command line editing   No   No    Yes   Yes    Yes

job control            No   Yes   Yes   Yes    Yes

________________________________________________________

For more information on the various features of each shell, take a look at the following web site.

#!/bin/sh, protection, simple commands

The rest of this lecture will cover basic mechanics of using the Bourne shell (/bin/sh). Even though most people use csh as the interactive interpreter to execute programs, many find the Bourne shell to be much simpler for writing scripts. The general opinion is that equivalent csh scripts often cannot be produced without a lot of pain and agony.

The man-page for the Bourne shell (``man sh'')is excellent. After these two lectures , you should be able to read it without problems, in order to learn things not covered. Please create a working directory and copy into it all the files in /home/cs300/Sh. 

Example:
UNIX> cd ~
UNIX> pwd
/home/myname
UNIX> mkdir cs300
UNIX> cd cs300
UNIX> mkdir Sh
UNIX> cp /home/cs300/Sh/* Sh
UNIX> cd Sh
UNIX> ls

Script

What is a script? The best analogy I have heard is that it is like the script of a play. Just a list of things to say and do. For us, a script is a file containing a list of text commands understood by the shell or some other command interpreter. Another concept is that it is a program that does not need to be compiled. So onward.

Writing a shell script

To write a Bourne shell script, the first line of your program should be ``#!/bin/sh''. Moreover, the protection mode of your program should have the executable bit set. You can check the your program's permissions using the long listing command. 
UNIX> ls -l
UNIX> chmod 0750 {filename}  or 
UNIX> chmod +x {filename}
This will set the executable bit on your program. After the first line, you may execute programs pretty much like in the csh. For example, the lshome program prints your home directory, and then lists its contents. Try it out: 

CS300> lshome

/lymon/homes/cs300
AdobeFnt.lst  Jgraph        Python        attend        mail
Awk           Mail          Sed           bin           www-home
Cat           Make          Sh            cs300.tar.gz
Expect        Misc          Tcl           grades
Gcc           Perl          archive       junk

CS300>

You can also run the shell script by typing ``sh lshome'' (and then you don't need the ``#!/bin/sh'' line). Doing ``sh -x script'' can help you debug a shell script. The -x switch tells the shell to print each command as it is executed so that you can see where the script fails. Try that with lshome. 


CS300> sh -x lshome
+ cd 
+ pwd 
/homes/cs300
+ ls 
AdobeFnt.lst  Jgraph        Python        attend        mail
Awk           Mail          Sed           bin           www-home
Cat           Make          Sh            cs300.tar.gz
Expect        Misc          Tcl           grades
Gcc           Perl          archive       junk
A last point on running the Bourne shell is that you can run it interactively by simply typing sh at a command prompt. This means that you can check out some of these things we are talking about without having to edit/create script files.

Indirection, pipes in sh

A little background on this next subject might help. In the C programming language, which some of you may be familiar with, there are numbers assigned to the streams stdin, stdout and stderr ; 0, 1 and 2 respectively. UNIX, and most of the shells that run on UNIX, are written in C. So there is logic behind the following.

If you have been using UNIX for any time indirection and pipes are probably second nature to you by now. If not here are the basic indirection cases:

So, suppose the file f1 contains the bytes ``This is f1'', and suppose that f3 does not exist. Then what happens when you do the following? 
$ cat < f1
$ cat f1 f3 > f2
$ cat f1 f3 2> f2
$ cat f1 f3 2>&1 > f2
$ cat f1 f3 > f2 2>&1
$ cat f1 f3 >&2 2> f2
$ cat f1 f3 2> f2 >&2
$ cat f1 f3 2>&1 >f2 | cat > f5
Make sure you understand the output of each of these (when you test it, make sure you're running sh and not csh).

Pipes "pipe" standard output of one command into standard input of another. Again, I assume that this is something you already know. For example, a simple way of printing the 5th line of the file f is to do the following:

UNIX> head  -n5  f  |  tail  -n1

(Yes, there are better ways of doing that).

We will take up here next week. I'll finish the Bourne shell and give the first lab assignment.