Paul E. McKenney – Is Parallel Programming Hard, And If So, Why?


Objective of presentation:

To advocate for the position that although parallel programming might be unfamiliar to many, it is inevitable and not as difficult as the doom-cryers would have you believe.  If done properly, work-a-day parallel programming requires perhaps 5% more training than does sequential programming.



In less than a decade, multicore hardware has made the transition from exotic to ubiquitous.  To those of you who fear and loathe parallel programming, I offer my condolences, but the settled fact of the matter is that parallel programming has irrevocably joined mainstream programming practice.  However, I can also offer some good news along with the bad.  The good news is that parallel programming is not all that much more difficult than sequential programming.  Of course, some additional bad news is that most people cannot deal even with sequential programming. This talk will discuss ways that we can not just cope, but actually thrive in this new multicore environment.


-Review of MIPS/clock-frequency trends.
-Parallel is here to stay: parallel hand-held battery anecdote.
-Pastiche of “parallel is inhumanly hard” quotes.
-But the universe is concurrent!!!  And people are too!!! (cartoon)  Additional examples.
-But just because concurrency is natural does not mean that concurrent programming is natural.  Especially given that -programming- does not seem to be natural!  Three obstacles: (1) theory of mind (2) common sense (3) fragmentary plans. (Auto-rental example — free upgrade due to being Linux kernel hacker.)

Most people are not going to undertake parallel programming, mostly because most people are not going to program period!!!
Other topics from the blog series will be chosen randomly and capriciously, as there will be time for only a few:

  • The Great Multicore Software Crisis is upon us, but we can learn from the 1970s/1980s Great Software Crisis.
  • Embarrassing parallelism isn’t.
  • Parallel programmers can be trained.  Without experience and/or proper training, high IQ is a negative.
  • Darwinian selection favored fear and loathing of parallelism, but the fitness in the past does not necessarily imply fitness in the future.
  • Code is like water.  It is easy to do something with a cupful of water, but not so easy to do the same thing with the Pacific Ocean.
  • Past serial-only design and coding decisions cannot be wished away.
  • Parallelism introduces global constraints that are not as prevalent in sequential software.  A global view of parallel source code is critically important.
  • You get what you measure.  So be careful what you measure.
  • Amdahl’s Law is a great tool for evaluating scalability. Too bad that performance is what really matters.
  • Tools.  Parallel tools.  We need them.
  • Validating parallel programs requires special care.
  • Don’t ask the janitor to remodel your building. This caution also applies to software, despite the fact that your speaker is an exception to this rule.

But there is hope: like the Great Software Crisis of the 1970s and 1980s, the Great Multicore Software Crisis will spawn the good, the fad, and the ugly.  The new ubiquitous multicore hardware will be available to countless millions of developers and users, which will fuel a huge burst of creativity that will shock and amaze all of us, especially those who still fear and loathe parallel software.  As in the past this creativity will tend to favor open-source projects: if two heads are better than one, just try ten thousand heads!

Target audience:

Parallel developers, sequential developers, academics, users, and most especially innocent bystanders caught in the crossfire.

Project homepage / blog: