Multithreaded Server C++, Socket Programming

Question

I am learning socket programming. I have built a client server program, where each client is handled by an individual thread in server program. But after a certain number of threads the system's performance will degrade. Now, to solve this, my task is to make my server handle multiple clients (say N connections) with a single thread, and only create a thread when there are more than N connections, and manage those N connections.

Say, 1 thread currently can deal with 200 clients, I will create another thread, when 201th connection arrives.

How to achieve this? And what is the proper way to manage those N connections?

Here is my code:

Client Code

#define PORT 9876

bool client(struct sockaddr_in* server_addr, int& socketDesc)
{
    memset(server_addr, (char)0, sizeof(sockaddr_in));
    
    sd = socket(AF_INET, SOCK_STREAM, 0);

    if (socketDesc < 0)
    {
        cerr << "Cannot open socket" << endl;
        return false;
    }

    server_addr->sin_family = AF_INET;
    server_addr->sin_addr.s_addr = inet_addr("127.0.0.1");
    server_addr->sin_port = htons(PORT);

    int n;
    if ((n = connect(socketDesc, (struct sockaddr*)server_addr, sizeof(sockaddr_in)) < 0))
    {
        cerr << "Connect Failed" << endl;
        return false;
    }

    return success;
}

int main(int argc, char* argv[])
{
    
    int socketDesc;
    struct sockaddr_in server_addr;
    
    client(&server_addr, socketDesc) > 0)
    
    const char* hello = "Hello from Client Side";
    

    for (int i = 0; i < 10; i++)
    {
        cout << hello << endl;
        send(sd, hello, strlen(hello), 0);
    }

    return 0;

}

Server Code:

#define PORT 9876

void server(struct sockaddr_in* server_addr, int& socketDesc)
{
    memset(server_addr, (char)0, sizeof(sockaddr_in));
    
    socketDesc = socket(AF_INET, SOCK_STREAM, 0);

    if (socketDesc < 0)
    {
        cerr << "Cannot open socket" << endl;
        return;
    }
    server_addr->sin_family = AF_INET;
    server_addr->sin_addr.s_addr = INADDR_ANY;
    server_addr->sin_port = htons(PORT);
    int n;

    if ((n = bind(socketDesc, (struct sockaddr*)server_addr, sizeof(sockaddr_in))) < 0)
    {
        cerr << "Cannot Bind" << endl;
        return;
    }

    if ((n = listen(socketDesc, SOMAXCONN) < 0))
    {
        cerr << "Cannot Listen!" << endl;
        return;
    }
}
void func(struct sockaddr_in* client_addr, int& new_sd)
{
    char memory[1024] = { 0 };
    int n;

    while ((n = recv(new_sd, memory, 1024, 0)))
    {
        cout << "Current Thread ID " << this_thread::get_id() << endl;
        cout << buffer << endl;
    }
    
}
    
int main(int argc, char* argv[])
{
    int socketDesc;
    struct sockaddr_in server_addr;
    server(&server_addr, socketDesc);

    vector<std::thread> poolofthread;
    struct sockaddr_in client_addr;
    socklen_t length = sizeof(sockaddr_in);
    int new_sd;

            
    while ((new_sd = accept(socketDesc, (sockaddr*)&client_addr, (socklen_t*)&length)) > 0)
    {
        cout << "Client Accepted" << endl;
        thread t(func, &client_addr, ref(new_sd));
        poolofthread.push_back(move(t));
    }
    return 0;

}

This Code for each client handled by separate thread is working fine.

I have tried handling each client with a single thread. Now, I want to handle N connections with a single thread. And only create a new thread when it is necessary(LIKE, when more than N connection arrives).

If you can give some idea about the best way to manage those N connections, then it would be helpful. I am learning select(), poll and epoll. How can I use them and which will be best with my case and why?

I have tried clearing the question and the problem I am facing. If you don't understand any part please write in the comments.

Answer 1

I highly suggest you to read some about different server architectures and design methods. I had the same problem as yours and found this amazing website that explains different types and architectures of servers along with their pros and cons and challenges. Pay attention that it has multiple parts. It'll start with beginning to design a server and at each step improves it.

For a short description, a good practice design is that there's a netIOManager (could be a ThreadPool) that will only receive net data and hand it over to a MainManager(also another ThreadPool) to process that data and then immediately return to its main job which is receiving net data.

The MainManager may have further database tasks(also another ThreadPool) and then after processing the data it will deliver the response to the netIoManager so it will send the response to the client. here's a visualization

As you can see in this scenario you have 3 ThreadPools(services) that can be connected using task Queues between them(see producer consumer problem)

If you search the internet there are many implementations of thread pool for C++.

Also for netIOManager part again i recommend you to read that site i mentioned earlier. The event-driven solution is the best i have found so far

Good luck!