Reduce topic replication factor with Kafka manager or Kafka cli

Question

There are currently 22 replicas configured for specific topic in Kafka 0.9.0.1.

Is it possible to reduce the replication factor of the topic to 3?

How to do it via Kafka CLI or Kafka Manager?

I found a way to increase replicas number only here

Answer 1

Yes. Changing (increasing or decreasing) the replication factor can be done using the following 2-step process:

1. First, you'll need to create a partition assignment structure for the given topic in the form of a json file. Here's an example:

   {
    "version":1,
    "partitions":[
         {"topic":"<topic-name>","partition":0,"replicas":[<broker-ids>]},
         {"topic":"<topic-name","partition":1,"replicas":[<broker-ids>]},
         ...
         {"topic":"<topic-name","partition":n,"replicas":[<broker-ids>]},
    ]
   }
   

   Save this file with any name. Let's say - decrease-replication-factor.json. Note - The <broker-ids> in the end represents the comma separated list of broker ids you want your replicas to exist on.

2. Run the script kafka-reassign-paritions and supply the above json as an input in the following way:

   kafka-reassign-partitions --zookeeper <zookeeper-server-list>:2181 
   --reassignment-json-file decrease-replication-factor.json --execute
   

Now, if you run the describe command for the given topic, you should see the reduced replicas as per the supplied json.

There are some tools as well created in the Kafka community that can help you achieve this. Here is one such example created by LinkedIn.