Concurrent bear – parallel fish
Often when writing high performance code one can fall into the trap of making assumptions about the nature of the problem. Donald Knuth said “Premature optimisation is the root of all evil”. If that is the case then prejudiced optimisation is the seed.
I learned the bitter lesson myself. Recently while building an architecture to support crunching what I thought was a moderately large amount of data I made the mistake of designing it with multiple threads of execution. At the beginning my rationale seemed quite sound as many have postulated that the performance gain to be had from single threaded execution of code is limited. The architecture that I was building involved a fast in memory key value store called Redis as one of it’s pieces. Unfortunately in the case of Redis the opposite was true. Because it’s often expensive to synchronise between threads and because in the normal use case for Redis the bottleneck was the network and not the CPU it’s authors decided to make it a single threaded application taking and serving requests off the wire one at a time. However it can handle multiple clients at a time, that is it is a program that supports concurrency but that’s not the same thing as parallelism.
To sum up in brief, concurrency is about breaking up a problem into components that can execute independently. They may not execute together at the same time – that’s the domain of parallelism. A parallel implementation of a concurrent problem is capable of executing multiple concurrent things at the same time. Now that your further confused…
Redis was not able to handle all the parallel execution of the rest of the system in that it was only able to serve one client at any given point of time, even though it could serve multiple clients concurrently. This created a pile up on the Redis thread that handled the execution of commands to Redis, consequently blocking the lovely parallel design of the rest of the system. This resulted in a lot of thumb twiddling for a lot of CPU cores running everything else.
The moral of the story of course is that you should never make assumptions on the nature of a problem. Sometimes the best solution to a problem may go against the flow of existing wisdom. It’s always best to start simple and find out about the nature of the problem as you go along rather than to start complex and have to throw a lot of pretty code away… as I had to.