grpc server handles multiple requests in parallel

Question

I have a question regarding the grpc server handles multiple requests in parallel, I have a grpc server, and the server provides an endpoint to handle client requests, and there are multiple clients sending request to the same endpoint.

When different clients send multiple requests to server at the same time, how the server handle those requests received the same time? Will each request will be handled by a thread simultaneously? Or the requests will be queued and handled one by one?

Thanks!

Answer 1

HTTP/2 connections have a limit on the number of maximum concurrent streams on a connection at one time. By default, most servers set this limit to 100 concurrent streams.

A gRPC channel uses a single HTTP/2 connection, and concurrent calls are multiplexed on that connection. When the number of active calls reaches the connection stream limit, additional calls are queued in the client. Queued calls wait for active calls to complete before they are sent. Applications with high load, or long running streaming gRPC calls, could see performance issues caused by calls queuing because of this limit.

But this problem has its own solution, for example in .Net, we could set following setting while defining GrpcChannel:

SocketsHttpHandler.EnableMultipleHttp2Connections = true

and it means, when the concurrent stream limit is reached, create additional HTTP/2 connections by a channel.