Single Vs Multi-thread Model

As a beginner, when we hear a Single thread model being more scalable compared to the multi-thread model, it sounds quite confusing ! How can a team be less powerful compared to a one man army ?

Well, even though it may not be true always, but it’s definitely true in handling high volume data intensive requests. By data intensive request we mean, calls involving more of IO calls (calls to databases or services etc) and requiring minimal cpu processing.

Moreover, Nodejs has been designed to handle this specific scenario better. And, it is one of the key reason behind its early acceptance and rapid growth in top tier internet companies.

In this article, we will focus on this particular feature to see what problem it solved and how ?

A Typical Data Intensive Request

Let’s consider a bank account home page where we would be showing customer account summary. We may just use the client id to get the summary as follows:

  1. Fetch his account numbers using the client id and use this to fetch the following details
    1. Get his savings account balance
    2. Fetch his latest transactions using
    3. Fetch his fixed deposit balance
  2. and return an account summary.

As we can see such types of request require more of IO calls compared to any heavy computational work.

In most of our applications that we use on daily basis, the requests mostly involves CRUD operations and very minimal processing as in the example given above.

So, let us consider such a data intensive request where the IO calls share the majority of the response time as shown below.

  • Three very small parts needing CPU processing and
  • Two much longer parts involving IO calls.

Assuming we have a continuous flow of this call from different clients, lets see how the two systems will handle them.

Multi-thread Model

In a multi-threaded system, each thread takes up call to process it end to end before its ready to accept a new one. Hence, besides doing its part of processing, it also waits on the IO calls for the response to come back.

As we can see, even if we have multiple threads working on the incoming requests, there are many a times all of them sit idle.

CPU Idle Time in Multi Thread Models

When all threads are waiting it simply wastes the CPU time, even if there would be many new request in the pending queue. If this case continues longer, it will keep building up the pending queue, hence the response time and, finally, result in a server crash.

In other words, we are under utilizing the power of the our CPU and would be blaming him for the under performance. The Single thread model tries to do away with this injustice and allows the CPU to show case its capability in full.

 

Single Thread Model – Reactive Models

In the Single thread model we are ensuring the main thread is not waiting on these time consuming IO calls.

Concept behind NodeJS Single Thread Model

It provides asynchronous IO APIs and a small number threads in a worker pool, so that the main thread can offload the waiting part.

As soon as it hands over the IO call to a thread in worker pool, it proceeds with other tasks pending for it. Meanwhile, as the blocking IO returns the response, the thread registers the associated callback into the NodeJS Event Loop. The callback defines the further processing; using the result from the asynchronous call. The processing of the request; sharing it between the main thread and worker pool then looks as shown in the below diagram.

Sharing of Request Processing between the Main Thread & the Worker Threads

 

Importantly, as we can see, the main thread now only works on what needs it attention and never waits on others. This maximizes the CPU utilization and, hence, the scalability.

  • In our example, whereas with 10 threads we can process 10 request in 4 seconds using multi-thread model; we can process around 100 requests within the same time using the single thread models.

The below diagram shows another view of what we have seen above. To be specific, it shows how we divide the parts of a request between the main thread and the worker pool using the even loop.

Request Processing in Nodejs using Event Loop & Worker Threads

 

Summary

Nodejs is quite effective for handling high volume, data intensive calls with significantly less number of servers. By data intensive calls, we mean the calls involving more of IO calls and minimal CPU processing. It can provide a much better performance in terms of response time and handling spikes in the load.

Because of this feature, many high volume internet based companies such as LinkedIn, Uber, Netflix have been early adopters of NodeJS. High scalability, better load handling, usage of the same language for both front-end and back-end are the some of the key reasons for making NodeJS so popular across the industry, so fast.

We should remember that the server was not built for CPU intensive tasks such as complex calculations, image or video processing etc. Hence, we should always keep such stuffs away from the main thread, ensuring we do not block it for long. For such tasks, we should make use of the child threads which was introduced later in version 10.5.0 for optimal processing.