CS594 Spring 2002

Performance Analysis Tools

Homework Assignment

Due Wednesday, April 10

As members of a scientific computing group at a large high performance computing center, you have been asked to evaluate the following set of performance analysis tools for use on a large Linux/x86 cluster:

• DEEP/MPI
• MPE Logging/Jumpshot
• Pablo Performance Analysis Tools
• Paradyn
• Scalea
• TAU
• Vampir
• Vprof

Each of you will be assigned one of these tools to evaluate. In addition to any examples provided with the tool, you should attempt to use one or more of the Using MPI intermediate examples (available from ftp://info.mcs.anl.gov/pub/mpi/using) as test programs. For a large-scale program, use one of the NAS Parallel Benchmarks with a large problem size (available from http://www.nas.nasa.gov/NAS/NPB/). You should download and install your assigned tool on the TORC cluster and answer the following questions:

1. Installation

1.1 What additional software packages does the tool require?

1.2 How difficult was the installation and how long did it take for you to install the tool?

1.3 Was adequate documentation and (if needed) support provided?

1.4 Any additional comments about the installation

2. Ease of use

2.1 How much effort is required to collect performance data? How long did it take to successfully instrument the test programs and collect performance data? Is instrumentation automatic or does it require manual modification of the source program?

2.2 What is the quality of the documentation provided?

2.3 Were example programs and/or performance data provided? If so, how effective were they at illustrating use of the tool?

2.4 Any additional comments about ease of use

3. Functionality

3.1 What levels of support for profiling are provided – e.g., none, subroutine level, basic block profiling?

3.2 Is MPI trace record collection provided?

3.3 What types of analysis are supported?

3.4 Any additional comments about functionality (e.g., support for mixed MPI/OpenMP program analysis, support for heterogeneous and distributed environments, analysis of MPI-2 I/O)

4. Robustness

4.1 Did the tool operate correctly without crashing?

4.2 If errors occurred, were they handled gracefully with appropriate error messages?

4.3 Any additional comments about robustness

5. Scalability

5.1 Does the tool provide mechanisms for controlling the amount of performance

data generated by large programs and/or for managing large amounts of performance data?

5.2 Are the analysis tools able to display data for large numbers of processors?

5.3 Additional comments about scalability

6. Portability

6.1 What processors, operating systems, languages, and compilers does the tool work with?

6.2 Other comments about portability

7. Versatility

7.1 Can the tool be used to do different types of data collection and analysis? ((If so, please describe)

7.2 Can the tool interoperate with other tools? (If so, please describe)