Λ10 - Παραλληλος και Κατανεμημενος Προγραμματισμος
(Parallel and Distributed Programming)
Fall 2009

This aim of this course is to teach the various models and practices employed for programming shared-memory multiprocessors, multicores and distributed-memory multicomputers as well as some of the newly popular platforms such as (GP)GPUs. Students will be called to study papers related to the topics covered in the course as well as program on shared-memory machines, on clusters and on simulators of specialized hardware. The course covers:
  • Introduction to shared-memory and distributed-memory architectures and the characteristics a programmer should care about.
  • Shared address space programming models and facilities: processes, (POSIX) threads, OpenMP, OpenMP tasks, cilk, user-level threading
  • Distributed programming: RPC, MPI, advanced MPI, software distributed shared memory (sDSM)
  • Assorted extra: (GP)GPUs, Cell, CUDA

Student Projects / Presentations:

  • GPU programming (CUDA, OpenCL)
  • Lock-free and related algorithms for queues and deqeues.
  • MPI-IO and parallel file systems

Reading Assignments

Reading Assignment (due Dec. 15, 2009)

   Software DSM implementations

Programming Assignment (due Dec. 8, 2009)

Reading Assignment (due Dec. 1, 2009)

   Work stealing & Cilk-5 implementation

Reading Assignment (due Nov. 17, 2009)

   Transactional memory

Reading Assignment (due Nov. 3, 2009)

   Algorithms for locks and barriers

Course material / Other resources

Lecture slides (pdf)

(Access only from departmental machines)

Some free books

Some textbooks:

  • B. Wilkinson, M. Allen, Parallel programming: Techniques and applications using networked workstations and parallel computers, Prentice Hall (1st ed: 1999, 2nd ed: 2004)
  • A. Grama, G. Karypis, V. Kumar, A. Gupta, Introduction to Parallel computing (2nd ed), Addison Wesley, 2003
  • R. Chandra, et al, Parallel programming in OpenMP, Academic Press (2001)

Other stuff