Introduction to Parallel Computing and Scientific Computation

21-765. Introduction to Parallel Computing and Scientific Computation

Location: Wean Hall, Room 6403b Time: TBA
Next Lecture: TBA

Level: Introductory Coverage: General

Instructor: Florin B. Manolache
Objectives Slides, Homeworks

e-mail: florin@andrew.cmu.edu
General Considerations Projects

Office: Wean Hall, Room 6108

Curriculum Bibliography

Office time: 9am-3pm
Credit Administrative

The objectives of this course are:

to develop structural intuition of how the hardware and the software work, starting from simple systems to complex shared resource architectures;
to provide guidelines about how to write and document a software package;
to familiarize the audience with the main parallel programming techniques and the common software packages/libraries.

General Considerations

The course is intended to be self-consistent, no prior computer skills being required. However, familiarity with the C programming language and Unix command line should give the student more time to concentrate on the core issues of the course, as hardware structure, operating system and networking insights, numerical methods.

The main idea of the course is to give the student a hands-on experience of writing a simple software package that eventually can be implemented on a parallel computer architecture. All the steps and components of the process (defining the problem, numerical algorithms, program design, coding, different levels of documentation) are treated at a basic level. Everything is done in the context of a structured vision of the computing environment.

The typical programming environment makes the computer hardware and operating system transparent to the user. In contrast, each program intended for efficient parallel execution must consider the custom physical and logical communication topology of the processors in a parallel system. The course should give a general image over the entire range of issues that a developer should consider when designing a parallel algorithm, from principles to details. The knowledge provided by the course should be enough to help the audience decide what's the most appropriate technique to approach a problem on a given computer architecture. However, the development of an efficient algorithm will require a lot of additional study, practice, and experimental work.

The examples, exercises, and projects were determined by the computers and software available for practice. The following were preferred: the C language, the i386 (PC) hardware platform, and the Linux operating system. However, the presentation will be kept at a very general level such that the student is prepared for any actual parallel computing environment.

The course contains three parts:

The first part makes the connection between real life and the computer world.

Chapter 1 shows how to transform a real life problem into a sequential computer algorithm, with reference to basic numerical methods.
Chapter 2 describes a software package and presents design, development, and maintenance concerns.
Chapter 3 contains basic programming notions with applications to the C language, libraries, compilers, debuggers.
Chapter 4 describes a model of structural information organization with applications to linear media and filesystems.

The second part provides the background needed to understand how the computer systems work.

Chapter 5 presents a layered model of the computing world.
Chapter 6 gives elementary notions of binary logics and digital electronics.
Chapter 7 the layered model of the computer hardware basics; application: how to build your own computer.
Chapter 8 presents a typical operating system, explores user interfaces, other user level issues.
Chapter 9 describes computer networks, topology, and layered communication protocols.

The third part explores the performance computing world.

Chapter 10 presents the parallel computing basic concepts and programming techniques.
Chapter 11 discusses the principal parallel architectures and the characteristics of the typical software packages.
Chapter 12 explains how pvm works.
Chapter 13 discusses the MPI standard, one common implementation, additional libraries issues.
Chapter 14 shows how to take advantage of multiple processors using SMP.
Chapter 15 presents modern approaches to resource sharing at different scales: mosix, condor, GRID.

Credit:

Students will have to pick a project no later than the third week of the semester and to deliver milestones every other week. A project consists in developing a (simple) software package or module that have a defined practical purpose. More details and alternatives are here. Students are welcome to discuss with the instructor projects close to their scientific interests, or pick one of the offered projects.

Grading is done based on class attendance and merits of the project as:

how well is the program structured such that it will allow easy further development, easy debugging
the quality (and not the length) of the documentation
how functional, efficient, and/or innovative is the numerical algorithm (tested on the provided examples)

Short homeworks will be assigned during each lecture. Submitting solutions is not mandatory, but can be considered for grading. The purpose of the homeworks is to help develop practical skills and get used with the computing environment. The homeworks are recommended for students auditing the lectures as well.

Administrative

Students are welcome to participate for credit or for fun.

The time of the lectures will be established by consulting the schedules of the students interested in the course. Registered students' schedule have priority. The time will be announced on this webpage and by e-mail ASAP.

Unregistered students should express their interest and time preferences by e-mail to florin@andrew.cmu.edu or in person (Wean Hall, Room 6108) at any time before second day of school of the Spring Semester.

I'm always available for lengthy consultations and discussions regarding the projects and the curriculum.

Instructor: Florin B. Manolache	Objectives	Slides, Homeworks
e-mail: florin@andrew.cmu.edu	General Considerations	Projects
Office: Wean Hall, Room 6108	Curriculum	Bibliography
Office time: 9am-3pm	Credit	Administrative