The computer that I am typing this on has 10 cores. The computers that you use every day, whether it is for work or it is inside of your phone, all have multiple cores. Consumer, desktop-level CPUs may have as many as 16 or 18 cores in them, and these numbers continue to increase as technological advancements permit greater numbers of transistors on each processing unit. Multicore processors are the norm, now, but it was not so long ago that everything was very different.
While there have been commercial-scale computer systems equipped with multiple processors for decades, and there were motherboards for PC systems that allowed multiple CPUs as early as the 1990s, most computers that most people used were only equipped with a single processing unit through the 1990s. Those early multiprocessor systems were generally restricted to workstation-level systems.
In 1999, there was a shift. ABIT released a motherboard that allowed two relatively inexpensive, commodity Intel Celeron processors to be used together. This ushered in an era where more and more systems were sold with two processors. The selling point to this was simple -- even if you were doing something intensive with the computer on one CPU, it had another that was available for other work. However, most programmers, and the languages that they used, focused on writing software that ran on only a single processor.
When one wanted to get more work done than could be done with a single process running on a single processor, one would start another process. This approach is referred to as multiprocessing.
Then, in 2005, the world shifted. Both AMD and Intel released multicore CPUs. These were CPU chips that had more than one processing unit housed inside of the same chip. Prior to this, the arms race among chip manufacturers had focused on increasing the speed of successive generations of processors, but after multi-core processors were released, the growth in CPU capabilities took on a new dimension. Two core processors were replaced with four core processors, and four-core processors were replaced with six-core processors.
One core, two cores, four cores, eighteen cores....what does it matter? What are cores and why should you care?
A core is a processor. Multiple cores mean that the computer can do multiple things in parallel. This is important because while doing things in parallel means that the computer can do more total work in any given period of time. For example, think about a program that takes requests to calculate the factors of very large numbers. This task is computationally expensive. That is, it requires a lot of calculation in order to deliver a result.
If a computer has a single core, then while it is working on this task, it will be unable to do anything else unless it takes time (and effort) away from the main task. It will also be unable to do more than one of these calculations at a time. If the computer has multiple cores, though, it could use one of them to work on the factorial calculation, while the other remains free to respond to your cursor movements and mouse clicks so that you can read Twitter while you wait.
If a computer has many cores, it might be able to figure out half a dozen factorial calculations in the time that a single core figures out one. This is the benefit of parallel operations on multiple cores. If it helps, think of each core as a person. There are some things where one person is enough to get the work done, or where it is faster to have one very fast, skilled person do the job than to try to divide it amongst multiple people, but most of the time, more work will get done when each person in a group has a job that they can do.
You may have noticed that the title is "Let's Talk About Concurrency", but in the preceding paragraphs we have been discussing cores and parallelism. If that has left you wondering if concurrency and parallelism are the same thing, you would not be alone. So, let's clear that up.
Imagine that you are in the kitchen, and that you are preparing a meal. There are a large number of tasks that are required while you prepare that meal. You have to gather the ingredients, and you may have to cut or chop some of them. You may have to put something in the microwave to warm or defrost, and while that is happening, maybe you do something else. Eventually, you put something in the oven, or on the stove, and while it is cooking, you are cleaning up your mess, or you start working on the tasks for the next item in the meal.
That is an example of concurrency. You are only ever doing a single thing at a time, but you may have multiple tasks that you are working on together, and you switch from focusing on one to focusing on another. You also have tasks where you are required to wait on something outside of your control -- the microwave or the oven, in this example -- but rather than just stand by, doing nothing, you use that time waiting to do something else, only coming back to your pot on the stove once it begins to boil, or once the food needs to be taken off the heat.
This juggling of many different tasks in the same general timeframe, switching between them as necessary, and sometimes even changing the order of those events somewhat -- next time, maybe you chop the jalapenos before you put the ground beef into the microwave to thaw, instead of after -- is concurrency in real life.
To put it another way, concurrency is when you can break down a complex task into multiple smaller tasks that can be done in the same general time period, ,with the order of events often changing from one run to the next, while still getting the same result in the end.
Parallelism would be what you would get if you added a second cook to the kitchen. Both cooks could be working on different parts of the dish at the same time. Each cook would have it's own concurrent set of tasks to perform, but both cooks could be doing those things together, at the same time, and theoretically the dish might be finished in half the time, or the cooks might be able to make two dishes in the time that each individually could only make one.
Two decades ago, most personal computers had a single core/CPU. Servers that internet sites, that commercial applications, and that commercial databases ran on might have two or even four discrete CPUs (and in some cases more, though those were extremely high end machines). Because of this lack of discrete processing units in computers of the time, programming languages of the time often spent little or no time really focusing on features that made use of concurrent workloads or parallelism.
For example, the first internet startup that I worked at operated in the email space. We basically published something kind of like a cross between a newspaper/magazine with personalized web based content, via HTML email, for subscribers. Are you interested in stocks, news, or sports? Do you want TV listings for all of the shows on all of the channels that interest you? We had an email product that would bring you just that content. The difficulty with this was one of resources, in an era when servers were vastly less powerful than just about anything that anyone has at their disposal today So, if you had a quarter of a million TV listings emails to deliver, and your extremely fast, extremely optimized (Perl) build could build 60000 emails an hour, how do you get everything built and mailed in less than an hour?
You did it, not by running a big parallel build on a multicore machine, as might happen today, but rather just by splitting the task into four or five processes, running across totally separate (single CPU) systems, each handling part of the workload. Your concurrency and your parallelism were features of how one used the software, operating system, and hardware, and were not things that the implementation language concerned itself with.
It was in this era of computers and programming languages that Ruby was formed, and that legacy has brought with it an interesting evolution to Ruby's concurrent and parallel programming features, from the green threads of early Ruby to the Ractors and asynchronous fiber schedulers of Ruby 3.0, with a number of interesting evolutions along the way.
Come back next week where we dive into some practical examples showing how to write Ruby code to handle concurrent and parallel workloads using multiple processes. This is the oldest approach to managing concurrent and parallel workloads, but despite that, it is still sometimes the best approach of all of the ones available.