Lab #1:
Introduction to MPI on Clusters of Workstations

Parallel Programming with MPI on Clusters of Workstations
CS 430, Fall 1999
Due: November 4, 1999

Purpose

The purpose of this lab is to become familiar with MPI account setup and documentation as well as compiling and running elementary to intermediate level MPI programs on clusters of workstations.

This lab and those to follow are self-contained, but you might find it helpful to see the section "Documentation for MPI and mpich."

Since cs 430 lacks a lab time please be sure to take advantage of the TAs' office hours and/or emailing them with questions.

Be sure to include a header with your full name, user name, program name, and date. Also, all MPI functions should be commented explaining what they are doing.

Lab 1 consists of 2 programs and should take less then an evenings amount of work. Commenting will be expected. Specifically, we will be looking for comments on mpi functions describing their actions and use.

Contents

  1. Introduction to MPI on Clusters of Workstations
    1. Workstation Login and Environment Setup
    2. MPI Programming - Exercise Information
    3. Submitting the programs
  2. Programs for Lab 1
    1. Hello World 1 - The Minimal MPI Program
    2. Hello World 2 - Hello Again
  3. Other Important Information
    1. Documentation for MPI, mpich, and Additional Resources

Workstation Login and Environment Setup

For your convenience, MPI has already been installed on the UTK CS Cetus machines. On any other system, please ensure that some implementation of MPI is installed before proceeding. We will be testing your programs on the Cetus machines so I strongly suggest that you test your programs there.

  1. Assuming that you all are using CS accounts, you may need to change your .cshrc file to access MPI. To do so, add the following information to your .cshrc file if you are using the C shell, or the equivalent if you are using another shell.

    2. MPI Programming - Exercise Information

    In this lab you will utilize the most fundamental MPI calls necessary for development of any MPI code, as well as learn how to compile and run MPI code. During this lab you will encounter the following exercises:

    1. Hello World 1 - a minimal MPI program (hello1 directory)
    2. Hello World 2 - Single Program with Multiple Data (hello2 directory)

    But first, the following . . .

    Submitting the program

    After you finish both programs in lab 1, please put them in a directory, and make sure there is no other files but your source files (or if you like, a README file). Then, type at the command prompt:

    unix> ~bbaker/bin/submit

    If you have any problems with submitting your code, please contact the TAs before the Lab due time.

    Hello World 1 - The Minimal MPI Program

    The objective of this exercise is not to demonstrate the fundamentals of compiling an MPI program and submitting it via mpirun.

    1. Examine the "Hello World!" program hello1.c.

      Notice that every process should print "Hello World!" and that the "Hello World!" program:
      1. Includes a header
      2. Initializes MPI
      3. Prints "Hello World!"
      4. Finalizes MPI
    2. Before writing hello1 lets review some MPI basics you will need to set up and know before going on.
      When compilling, the version of MPI that we are using (mpich) there are several ways to compile a program. We will use the commands mpicc to compile our C programs. These compilers locate the MPI libraries and header files as needed. Compile your programs from the unix prompt instead of using a Makefile.
      1. To compile a program called filename.c, at the command line enter the following:

        2. % mpicc -o filename filename.c

        where filename is the resulting executable.

        The resulting executable will be filename.

      So for the "Hello World!" program you will enter:

      % mpicc -o hello1 hello1.c

    3. Run "Hello World!". Since a parallel program runs on several processors, there needs to be some method of starting the program on each of the different processors. This method can vary from machine to machine, the mpich implementation of MPI provides a simple yet consistent way of starting programs, which hides all the implementation differences from you. This implementation is called mpirun. Job submission via mpirun typically takes the following form:

      4. % mpirun -np <num_of_procs> <program name>

      where <num_of_procs> is the desired number of processes and <program name> is the name of the executable compiled with mpicc.

      The list of machines used by mpirun is either a default list established by mpich or is a list created by you, the user, and specified at the command line with the -machinefile option. We will use both to submit the "Hello World!" program.

    4. Run "Hello World!" on the default set of machines. The default set of machines established by mpich can be found in the $MPIR_HOME/util/machines directory in the file machines.solaris.

      5. When using the default set of machines, a machine filename does not need to be specified with mpirun. Examine the default list of machines:

      % cat ${MPIR_HOME}/util/machines/machines.solaris

      You should see a list of machines present in the Cetus Lab.

      To see how mpirun assigns processes, we will use the -t option. This is the test option. It allows you to print what would be executed without actually running the program. NOTE: This option is not typically used when submitting a program, and is used here for illustrative purposes only.

      1. Run "Hello World!", requesting three processes:
        % mpirun -np 3 -t hello
        You should see something similar to the following output:
        Procgroup file:
        cetus3d 0 /copper/homes/user_name/lab1/hello1
        cetus4f 0 /copper/homes/user_name/lab1/hello1
        cetus2b 0 /copper/homes/user_name/lab1/hello1
        /copper/homes/user_name/lab1/hello -p4pg
        /a/cosmos/copper/homes/user_name/lab1/hello1/PI19505

        2. The "Hello World" program above was started on cetus3d. You should see the name of the machine you are on listed first as well. (If you wish to start a job, not using the local machine, then you may use the -nolocal option. See the mpirun man pages for more information.)

        The 0 following cetus3d indicates that this processor is the master. The path name immediately following the 0 lists the executable and the directory in which it resides. The next two lines indicate where the other two processes would have been assigned had the program actually been run via mpirun.

        The last two paths are first, a repeat of the program path that is to be run, and second, the residence of the output file that might be generated in conjunction with the -t option. If present it should start with PI and be followed by a number.

        How did mpirun assign the processes?

      2. Run "Hello World!" on 5 processes:

        3. % mpirun -np 5 -t hello

        Again, how did mpirun assign the processes?

      3. Run "Hello World!" requesting one process:

        4. % mpirun -np 1 -t hello

        How was the process assigned?

    5. You could specify the desired machines to use by placing a Machines file in your directory and specify if at compile time. Run "Hello World!" utilizing the -machinefile option. A machine file, called Machines, exists in the current directory. Examine it using (% more Machines) or by invoking your favorite editor. Notice that each machine must appear on its own line in order for mpirun to successfully start the processes. At this point you can either use the provided Machines file or create your own. If you create your own, and are in the UTK CS Cetus Lab, choose only three machines from the row you are sitting in. This will be important for the following exercise.

      6. Again, we will use the -t option to examine how mpirun allocates processes.

      1. Run "Hello World!" with the -machinefile option requesting one process:
        % mpirun -np 1 -machinefile Machines -t hello
      2. Run "Hello World!" with the -machinefile option requesting two processes:
        % mpirun -np 2 -machinefile Machines -t hello
      3. Run "Hello World!" with the -machinefile option requesting five processes:
        % mpirun -np 5 -machinefile Machines -t hello

      Notice that the Machines file has only three machines listed in it (more it again if you forgot). What happened when one process was requested? How did mpirun handle distribution of the process? How about when five processes were requested? How was distribution handled then? How did process distribution using the -machinefile option differ from distribution using the default machines list?

      Finally, for more information about mpirun see the man pages.

    6. Now that you are familiar with compiling and running an MPI program, modify the "Hello World!" program so that only the process ranked 0 in MPI_COMM_WORLD prints out ``Hello World!''. Compile and run it using either the default machine list or the created Machines file.
    7. The program you turn in should guarantee that every process prints its rank and the number of processes in the program, using the format "I am RANK out of SIZE". Additionally, be sure to place inline comments on all of the MPI functions. Commenting will be a part of your grade.

    Question: What happens if you omit the last MPI procedure call in your last MPI program? (It should hang.)

    Hello World 2 - Hello Again,

    The objective of this exercise it to become familiar with basic MPI routines used in almost any MPI program. You are asked to write an SPMD (Single Program Multiple Data) program where each process checks its rank, and decides if it is the master (if its rank is 0), or if it is a worker (if its rank is 1 or greater).

    1. The SPMD programs should:
      1. Include the header
      2. Initialize MPI
      3. Check its rank, and
        1. if the process is the master, then send a "Hello World!" message, in characters, to each of the workers.
        2. if the process is a worker, then receive the "Hello World!" message and print it out.
      4. Finalize MPI
    2. Compile your program at the command line. Run this code on 2 processes (% mpirun -np 2 hello2) using the default set of machines.
    3. Make the following modification to your program (this will be the version you turn in). After printing out the message, each worker sends a message containing a single integer, the workers rank, back to the master. The master should receive these integers and print out ``Answer received from RANK'' for each message received.


      4. Documentation for MPI and mpich, and additional resources

      1. There are man pages available for mpich which should now be installed in your MANPATH. Look at the following man pages to see some introductory information about MPI.

        2. % man MPI
        % man mpicc
        % man mpif77
        % man mpirun
        % man MPI_Init
        % man MPI_Finalize
        % man MPI_REAL

      2. There is a convenient X windows interface to these man pages that you may wish to leave running while you are working on your labs.

        3. % mpiman &

        The man pages are also available online.



      3. There is more information available for mpich located under the directory $MPIR_HOME. There are examples in the examples directory, and a users guide for mpich ( guide.ps.Z) in the $MPIR_HOME/doc directory.


      4. The WWW home for mpich is at the Argonne National Lab. Argon also maintain a page covering various MPI standards including MPI 1.1 and MPI 2.0. Here you can also find a general page on MPI standards including MPI version 1.1 in postscript (warning! this is 231 pages). Lastly on the mpich site you can find a helpful general tutorial on MPI as well as more advanced tutorials.


      5. An index of relevent MPI functions


      6. Credits: Thanks to JICS (Joint Institute for Computational Science) for materials from an Introduction to MPI Workshop.


      Bryan Baker & Rick Thursby
      Tue Oct 26 11:08:52 EDT 1999
      		Click here for solution courtesy of S. Chambers.