Types of Jobs
Probably the most important way of classifying jobs is by type of execution:
- Serial
- Multithreaded (a.k.a. single-node parallel, or shared memory parallel)
- Multinode parallel (a.k.a. MPI)
Central Processing Unit Hardware
Before talking about the execution models, knowing something of the underlying hardware will help. The execution models have to map onto the capability of the hardware.
The Central Processing Unit (CPU)[1] of a computer is the integrated circuit chip which executes instructions. Originally, CPUs were serial, i.e. they could run only one instruction at a time.
Modern CPUs have multiple "cores": each core is, in effect, a distinct CPU. These multiple cores are etched onto a single silicon chip so that the entire group may be considered one device. This aggregate device is known as a "socket", because the physical package is plugged into the circuit board into a socket.
Multiple cores means that the computer may execute more than one instruction simultaneously, i.e. parallel.
See Proteus Hardware and Software for a description of the hardware which makes up Proteus.
Even without multi-core CPUs, one may gain parallelism by distributing computation to multiple physical computers. This is commonly done using specialized software called MPI (Message Passing Interface). This provides a way for the programmer to deal with communication between different computers in order to coordinate a computation.
MPI may also be used on a single node with multiple CPU cores. The program treats each CPU core as a separate computer, while doing some simplification of the communication (since networking is not necessary for cores to transfer data between themselves).
MPI parallel execution may be enhanced by using multithreaded execution. This is called a "hybrid" model.
Execution Models
Serial
Serial execution means that the program runs one computer instruction at a time. This may also be known as "single-threaded". One can think of this as "reading order": one reads a block of text one word at a time.
Multithreaded
Multithreaded execution means that a program can run multiple threads of execution, each thread using a single CPU core for efficiency. While it is possible to run more threads than there are cores, doing so results in poorer performance since the "extra" threads would have to wait for a CPU core to be free to run some instructions. Running more threads than available cores is over-subscription.
Multi-node Parallel (MPI)
Multi-node parallel programs use network (or other types of) communication to coordinate a single large computation that has been distributed across multiple physical computers. A simple analog may be counting paper ballots by hand: many people (counters) are given subsets of the entire set of ballots, and each counter's totals are collated at the end.