21-765. Introduction to Parallel Computing and Scientific Computation

Location: POS 147   Time: Fri 2:00-3:50pm
First Lecture: Jan 19, 2024

Projects due: graduating students - May 5, 2024 ; non-graduating students - May 10, 2024

Level: Introductory        Coverage: General

Instructor: Florin B. Manolache
Objectives Slides, Homework
e-mail: florin@andrew.cmu.edu
General Considerations Projects
Office time: Wean Hall, Room 6218, 9am-3pm

Curriculum Bibliography
Course website: www.math.cmu.edu/~florin/M21-765/
Credit Administrativia

The objectives of this course are:

General Considerations

The course is intended to be self-consistent, no prior computer skills being required. However, familiarity with the C programming language and Unix command line should give the student more time to concentrate on the core issues of the course, as hardware structure, operating system and networking insights, numerical methods.

The main idea of the course is to give the student a hands-on experience of writing a simple software package that eventually can be implemented on a parallel computer architecture. All the steps and components of the process (defining the problem, numerical algorithms, program design, coding, different levels of documentation) are treated at a basic level. Everything is done in the context of a structured vision of the computing environment.

The typical programming environment makes the computer hardware and operating system transparent to the user. In contrast, each program intended for efficient parallel execution must consider the custom physical and logical communication topology of the processors in a parallel system. The course gives a general image over the entire range of issues that a developer should consider when designing a parallel algorithm, from principles to details. The knowledge provided by the course should be enough to help the audience decide what's the most appropriate technique to approach a problem on a given computer architecture. However, the development of an efficient algorithm will require a lot of additional study, practice, and experimental work.

The examples, exercises, and projects were determined by the computers and software available for practice. The following were preferred: the C language, the x86_64 hardware platform, and the Linux operating system. However, the presentation will be kept at a very general level such that the student is prepared for any real parallel computing environment. The individual study and the midterm project are based on Python.

The course contains three parts:

The first part makes the connection between real life and the computer world.
The second part provides the background needed to understand how computer systems work.
The third part explores the performance computing world.


Grading is based on three components:
Students will need to pick a final project no later than the third week of the semester and to deliver milestones every other week. A project consists in developing a (simple) software package or module that has a defined practical purpose. More details and alternatives are here. Students are welcome to discuss with the instructor projects close to their scientific interests, or pick one of the offered projects.

Merits of the final project considered for grading are:

Homework will be assigned after most lectures. Submitting solutions is not mandatory, but can add a bonus for the final grade. The purpose of the homework is to help develop practical skills and get used with the computing environment. The homework is recommended for students auditing the lectures as well. Students are encouraged to submit solutions containing interesting approaches or comments.