I think there is an issue about boundaries as to what is
ParallelProgramming. I had tried to be clear about it when I started
ParallelProgramming, but it seems that I have not been clear and that shows up as a problem worth discussing.
There are pages which link from
DistributedComputing which discuss issues of the patterns of programming. They include quite a lot of topics as can be seen from the fact that it has 56 back links.
I intended
ParallelProgramming to be about taking a serial program solving some numerical problem and turning it into a parallel program which will run on several computers in a closecoupled way. Maybe it is wrongly named, in which case it could be renamed something more specific. I now suggest
ProgrammingForParallelComputing. --
JohnFletcher
I think of
DistributedComputing as distributing a task among separate computers where it is expensive to share resources between nodes. I think of
ParallelProgramming as exploiting fine-grain parallelism in a problem, where sharing resources is easier. Examples:
MultiThreading to take advantage of
SymmetricMultiprocessing, and decomposing algorithms to use SIMD (
SingleInstructionMultipleData) to take advantage of vector units such as
AltiVec or
CellProcessors. --
IanOsgood
Here are some definitions from
WikiPedia.
On the page http://en.wikipedia.org/wiki/Parallel_computing.
Parallel computing is the simultaneous execution of the
same task (split up and specially adapted) on multiple processors in order to obtain results faster. The idea is based on the fact that the process of solving a problem usually can be divided into smaller tasks, which may be carried out simultaneously with some coordination.
A
parallel programming model is a set of software technologies to express parallel algorithms and match applications with the underlying parallel systems. It encloses the areas of applications, languages, compilers, libraries, communication systems, and parallel I/O. People have to choose a proper parallel programming model or a form of mixture of them to develop their parallel applications on a particular platform.
The items listed include ParallelVirtualMachine and MessagePassingInterface.
On the page http://en.wikipedia.org/wiki/Parallel_programming
Parallel programming is a
computer programming technique that provides for the execution of operations in parallel, either within a single computer, or across a number of systems. In the latter case, the term
distributed computing is used.
On the basis of the above I think that what I am discussing can be described as
ProgrammingForParallelComputing which has to use a
ParallelProgrammingModel. That has the virtue of being consistent with the
WikiPedia usage. --
JohnFletcher
Category and Concurrency
There are two categories
CategoryConcurrency and
CategoryConcurrencyPatterns.
There are also related pages called
SynchronizationStrategies and
ParalleliZation. There is some discussion about how these topics are related.
- Concurrency is when two tasks or threads are active at the same time.
- Synchronization is when the two tasks need to interact.
There is an Edit
Hint on
CategoryConcurrencyPatterns that
SynchronizationStrategies is the same thing and that they should be merged.
I think that ProgrammingForParallelComputing is a distinct area from the above which could have a category of its own but does not yet. I would welcome suggestions or contrary views. --
JohnFletcher
There is also the interesting book
PatternsForParallelProgramming.
CategoryDiscussion