Λ10 - Παραλληλος και Κατανεμημενος Προγραμματισμος
(Parallel and Distributed Programming)
Spring 2009

This aim of this course is to teach the various models and practices employed for programming shared-memory multiprocessors, multicores and distributed-memory multicomputers as well as some of the newly popular platforms such as (GP)GPUs. Students will be called to study papers related to the topics covered in the course as well as program on shared-memory machines, on clusters and on simulators of specialized hardware. The course covers:
  • Introduction to shared-memory and distributed-memory architectures and the characteristics a programmer should care about.
  • Shared address space programming models and facilities: processes, (POSIX) threads, OpenMP, OpenMP tasks, cilk, user-level threading
  • Distributed programming: RPC, MPI, advanced MPI, software distributed shared memory (sDSM)
  • Assorted extra: (GP)GPUs, Cell, CUDA

Announcements

  • 19/3/2009
    Κατόπιν αιτήματος από όλους τους φοιτητές, οι ώρες του μαθήματος αλλάζουν και από την επόμενη Τρίτη (24 Μαρτίου), οι διαλέξεις θα γίνονται:
        Τρίτη, 15:15 - 18:00

Projects / Assignments

Assignment 6

  1. MPI programming assignment: two implementations of the Sieve of Eratosthenes

Assignment 5 (due May 12, 2008)

  1. Cilk programming assignment

Assignment 4 (due May 5, 2009 -- reading assignment)

  1. Transactional memory

Assignment 3 (due May 5, 2009 -- double assignment)

  1. Multithreaded and OpenMP programming
  2. Find the maximum (and its position) element in an array
  3. Gaussian elimination
  4. A sample (compressed) 512x512 system of linear equations: A * x = b

Assignment 2 (due before Easter)

  1. Download, install and use OMPi.
  2. Compare ompicc, icc, suncc and gcc OpenMP overheads using the EPCC microbenchmarks (/usr/local/pub/epcc.tar.gz) for 2, 4, 6 and 8 threads.
  3. All experiments should be run in PARAGUAY (for login info contact the instructor).

Assignment 1 (due April 7, 2009)

  1. Implement a simple barrier for POSIX threads using condition variables
  2. Presentation of known barrier algorithms

Assignment 0 (due March 24, 2009 -- no credit)

  1. Study chapter 7 from my undergrad notes, on shared-memory programming. Also, go through chapter 2 which talks about shared-memory architectures.

Course material / Other resources

Lecture slides (pdf)

(Access only from departmental machines)

Some free books

Some textbooks:

  • B. Wilkinson, M. Allen, Parallel programming: Techniques and applications using networked workstations and parallel computers, Prentice Hall (1st ed: 1999, 2nd ed: 2004)
  • A. Grama, G. Karypis, V. Kumar, A. Gupta, Introduction to Parallel computing (2nd ed), Addison Wesley, 2003
  • R. Chandra, et al, Parallel programming in OpenMP, Academic Press (2001)

Other stuff