Syllabus – Math 4370/6370, Parallel Scientific Computing, Spring 2017

Class and Office Hours

Lecture: 156 Dallas Hall, M/W/F, 2:00-2:50 pm.

Office Hours: 139 Clements Hall, M/W/F 9-10, or by appointment (arrange by email).

Textbook and Web pages


Peter Pacheco, Introduction to Parallel Programming, Morgan Kaufmann, 1st edition, 2011. (ISBN: 9780123742605).

Highly Recommended:

Dietel, C++: How to Program, 8th edition, Pearson, 2011, ISBN 9780132662369.

Chapman, Jost & van der Pas, Using OpenMP, Portable Shared Memory Parallel Programming, MIT Press, 2007, ISBN 0262533022.

Gropp, Lusk & Skjellum, Using MPI – 3rd Edition: Portable Parallel Programming with the Message Passing Interface, MIT Press, 2014, ISBN 9780262527392.

Additional Reading (available in Math library or from instructor):

Shonkwiler & Lefton, Introduction to Parallel and Vector Scientific Computation, Cambridge University Press, 2006, ISBN 0521683378.

Supporting Materials (online):

Introduction to Parallel Computing – tutorial slides from LLNL

OpenMP – tutorial slides from LLNL

Message Passing Interface (MPI) – tutorial slides from LLNL

Designing and Building Parallel Programs – an online book by Ian Foster describing various concepts and tools for parallel software engineering.

Advanced MPI – tutorial slides from LLNL

Course Description

MATH 4370 – Parallel Scientific Computing [3 credits]

An introduction to parallel computing in the context of scientific computation.

Prerequisite: MATH 3316 and 3353, or permission of instructor.

MATH 6370 – Parallel Scientific Computing [3 credits]

An introduction to parallel computing in the context of scientific computation.

Prerequisite: MATH 5315 or CSE 7365 and MATH 6316 or CSE 7366.

Historically, parallel computing has addressed the demands of state-of-the-art engineering and scientific applications. Recently, however, other platforms including phones, video game consoles, laptops and desktops have transitioned toward chips with an increasing number of processor cores, requiring algorithms that may effectively utilize their parallel computing hardware.

The goal of MATH 4370/6370 is to introduce students to the foundations of parallel computing, including the principles of parallel computer architectures and parallel algorithm design. We will investigate a variety of algorithms, studying the relative benefits and weaknesses among different approaches for their parallel implementation. Students will learn how to use OpenMP and MPI to write their own simple parallel programs for shared- and distributed-memory systems, and will execute these programs on research computing clusters both here at SMU and at NSF supercomputer centers. Instruction will consist of both classroom lectures and computer lab tutorials.

Topics include:

  • Overview of scientific and engineering computing
  • Parallel computer architectures
  • Parallel programming models
  • Parallel programming with OpenMP and MPI
  • Advanced topics, including debugging parallel programs, performance analysis and optimization, parallel application packages, manycore programming, hybrid parallel programming and petascale computing (time permitting).

Students from all disciplines and computing background are welcome, as we will attempt to cover the necessary background from computer architecture, software, and numerical analysis. It is assumed that all students will have some experience programming in the C, C++ and/or FORTRAN programming languages.

Educational Outcomes

Upon successful completion of this course, students will have learned:

  • To identify different types of parallel computing hardware.
  • How to write shared-memory parallel programs using OpenMP.
  • How to write distributed-memory parallel programs using MPI.
  • The relative benefits of static and dynamic load-balancing within a parallel program.
  • Some intuition regarding which scientific computing algorithms may best utilize OpenMP and MPI parallel computing.
  • About multiple open-source software libraries for parallel scientific computing.


Students are expected to read either the book or online accompanying materials ahead of each lecture topic. Relevant sources and chapters/sections are identified on the lectures page.


In this course, homework and projects will make use of Linux workstations, clusters and supercomputers. Students who are unfamiliar with the Unix/Linux operating system should complete the following tutorial, to help them gain a basic understanding of Unix/Linux. Students who do not own Unix/Linux computers may follow the instructions setting up a Virtual Machine to run Linux within their existing operating system. Alternately, students who own Windows computers may follow the instructions setting up software to emulate and/or login to Unix/Linux servers. A good text editor for programming on both OS X and Windows computers is the Sublime text editor.

All students should request an account at the TACC User Portal (follow the “Create a TACC Account” link at the right of the page). In addition to SMU’s new research computing clusters, we will be using the NSF/TACC Stampede and Stampede-2 supercomputers (some of the largest in the world) in this class. Once you obtain your user account, email me with your username so that I can grant you access to our allocation of computing time.


As described below, students will be expected to submit programming assignments for their evaluation in the course. These assignments should be written in one of {C, C++, Fortran77/90/95/2003}. Further information on these languages may be easily found on the web, and I will be happy to discuss the relative merits of each language with students during office hours.

All students will perform their programming within the Git revision control system. Students should set up user accounts on Bitbucket – I recommend that you sign up using your SMU email account (since academic accounts get free perks). All course work should be performed within “private” repositories of your own making. In each of your repositories, you must grant me (drreynolds) read/write access. When each lab, homework or phase of your project is due, I will download the current state of your repository to do grading.


Homework assignments will be assigned periodically throughout the semester, and will consist of small programming exercises and written assignments. Homework problems must be completed on your own, though discussion of the assignments with other students and the instructor is encouraged. In addition, I encourage all students to visit my office when they get stuck on bugs in their codes.


During the first half (or so) of the semester, students will be expected to perform “lab” assignments. While these may be done with fellow students, they must be turned in individually. Instructions for each lab, along with the relevant files, are located on the course labs page.


You will each pursue a computing project throughout the semester. You will pick your projects during the first few weeks of the semester.

During each phase of the course, a portion of your project will be due (e.g. after we’ve finished with OpenMP, you will OpenMP-parallelize your project). These phases will include a one-on-one meeting with the professor to present your progress.

At the end of the semester (during the final exam period – May 15, 3:00) each student will give a short presentation to the class, detailing their project and results throughout the semester.

These projects will involve a substantial amount of programming – I highly recommend that you begin these early and do not procrastinate.


Students’ grades in this course will be determined based on the decomposition:

  • 20% homework
  • 20% labs
  • 60% projects

Honor Code

The SMU Honor Code applies to all homework and projects in this course. Work submitted for evaluation must represent your own individual effort. Any giving or receiving of aid without my express consent on academic work submitted for evaluation shall constitute a breach of the SMU Honor Code.

It is not possible to become a capable programmer without doing the programming yourself. This includes everything from initial program design all the way through debugging and finally generating results.

If you receive excessive assistance anywhere in this process, such that you would be unable to repeat the entire process yourself, then the work does not represent your own individual effort. Turning in such an assignment will violate the SMU Honor Code.

By the same token, providing excessive assistance to others is only a disservice to their learning, and will similarly be considered a violation of the SMU Honor Code.

As such, examples of honor code violations include:

  • Submitting homework results produced by someone else’s work (or that are made up).
  • Copying code from any non-approved source, e.g. online or from another student.
  • Supplying your own codes for another student to copy.
  • Debugging another student’s codes for them.

I take honor code violations very seriously, and will report all violations to the SMU Honor Council. The minimum penalty for a violation is a “0” on the assignment, and the maximum penalty is immediate failure of the course. These penalties are in addition to those imposed by the SMU Honor Council.

See the SMU Honor Code website for more information.

SMU Regulations

Disability Accommodations: Students needing academic accommodations for a disability must first register with Disability Accommodations & Success Strategies (DASS). Students can call 214-768-1470 or visit to begin the process. Once registered, students should then schedule an appointment with the professor as early in the semester as possible, present a DASS Accommodation Letter, and make appropriate arrangements. Please note that accommodations are not retroactive and require advance notice to implement.

Religious Observance: Religiously observant students wishing to be absent on holidays that require missing class should notify their professors in writing at the beginning of the semester, and should discuss with them, in advance, acceptable ways of making up any work missed because of the absence. (See University Policy No. 1.9.)

Excused Absences for University Extracurricular Activities: Students participating in an officially sanctioned, scheduled University extracurricular activity should be given the opportunity to make up class assignments or other graded assignments missed as a result of their participation. It is the responsibility of the student to make arrangements with the instructor prior to any missed scheduled examination or other missed assignment for making up the work. (University Undergraduate Catalogue)

Campus Carry: In accordance with Texas Senate Bill 11, also known as the “campus carry” law, following consultation with entire University community SMU determined to remain a weapons-free campus. Specifically, SMU prohibits possession of weapons (either openly or in a concealed manner) on campus. For more information, please see: