Implementation Mechanisms

Up to this point, we have focused on designing algorithms and the high-level constructs used to organize parallel programs. With this chapter, we shift gears and consider a program's source code and the low-level operations used to write parallel programs.

$\begin{lstlisting} ...$

We now drop the formalism of patterns. Most of the implementation mechanisms are included within the major parallel programming environments. Hence, rather than use patterns, we provide a high-level description of each implementation mechanism and then an investigation of how the mechanism maps onto our three target programming environments: OpenMP, MPI, and Java.

A complete and detailed discussion of these parallel programming ``building blocks'' would fill a large book. Fortunately, most parallel programmers use only a modest core subset of these mechanisms. These core implementation mechanisms fall into three categories:

	*UE management* Concurrent execution by its nature requires multiple entities that run at the same time. This means that programmers must manage the creation and destruction of processes and threads in a parallel program
	Synchronization Synchronization is used to enforce a constraint on the order of events occurring in different units of execution. The synchronization constructs described here include memory fences, barriers, and mutual exclusion
	*Communication* Concurrently executing threads or processes sometimes need to exchange information. When memory is not shared between them, this exchange occurs through and explicit communication event. The major types of communication events are message passing, and collective communication though we briefly describe several other common communication mechanisms as well.