How do you configure the java heap size for zookeeper?

Question

I am trying to set up Zookeeper 3.4.10 on an Ubuntu 18.04LTS Azure VM.

Per the ZooKeeper Administrator's Guide "incorrect Java heap size You should take special care to set your Java max heap size correctly. In particular, you should not create a situation in which ZooKeeper swaps to disk..."

The guide does not give instructions on how to set the max heap size. I've already done a little searching and found suggestions to create a java.env in the zookeeper/conf directory. I've done this and have tried setting the variable using two different methods I've found during research:

Attempt #1:

export JVMFLAGS="-Xmx6144m"

Attempt #2

#!/bin/bash
export CLASSPATH="~/zookeeper-3.4.10/conf/log4j.properties"
export JVMFLAGS="-Xmx6144m"

After making these changes, and restarting zookeeper, I checked the java heap size with:

java -XshowSettings:vm

And the max heap size is not changing.

What are the proper steps for configuring the max heap for zookeeper?

Answer 1

Binary release of Zookeeper contains bin directory with following files in which you interested in:

• zkServer.sh
• zkEnv.sh

zkEnv.sh defines location of all config files and some JVM tuning knobs, such as JVM heap size. JVM heap size can be changed by shell variable ZK_SERVER_HEAP(in MB).
Use following command to set custom heap size:

cd bin/
ZK_SERVER_HEAP=128 ./zkServer.sh start-foreground

In output of recent versions of Zookeeper you can find following lines:

2021-01-14 17:24:18,400 [myid:] - INFO  [main:Environment@98] - Server environment:os.memory.free=114MB
2021-01-14 17:24:18,400 [myid:] - INFO  [main:Environment@98] - Server environment:os.memory.max=128MB
2021-01-14 17:24:18,400 [myid:] - INFO  [main:Environment@98] - Server environment:os.memory.total=128MB

Other JVM opts can be set using SERVER_JVMFLAGS, for instance, use non default GC:

ZK_SERVER_HEAP=128 SERVER_JVMFLAGS="-XX:+UseShenandoahGC" ./zkServer.sh start-foreground