grpc call time reduce overhead

Question

I am using gRPC (non-blocking stub) in a java application and the response time between two function calls is around 5-8ms. I would like to reduce it. What do you suggest? Is it possible at all?

Answer 1

gRPC Java overheads are typically measured in µs (sub-ms). If you are seeing milliseconds of latency, the general expectations as to why:

1. Service latency is milliseconds. On the server, measure how long it takes the service to respond. Many services take milliseconds to process requests.

2. Network latency is milliseconds. Run ping from one host to the other to get a lower bound on communication performance. (netperf tcp_rr is an even better option if you have such interest.)

3. The benchmark did not warm up the JVM. The very first RPC will be very, very slow as class loading takes place. After that the JVM will use interpreted mode for a while before the JIT starts compiling code. The JIT will then progressively optimize code. So make sure to have a warm up period of 30+ seconds of sustained RPCs.

4. Channels aren't being reused. DNS lookups, creating network connections, and performing TLS handshakes increase latencies. But if you reuse Channels with multiple RPCs, then later RPCs won't need to pay those latency costs.