Kafka Connect HDFS Sink Issue

Question

I am trying to stream data using Kafka-Connect with HDFS Sink Connector. Both Standalone and Distributed modes are running fine but its writing into HDFS only once (based on flush-size) and not streaming later on. Please help if I'm missing some thing.

Confluent 2.0.0 & Kafka 0.9.0

Answer 1

I faced this issue long back.Just check below parameter is missing

Connect-hdfs-sink properties

"logs.dir":"/hdfs_directory/data/log"

"request.timeout.ms":"310000"

"offset.flush.interval.ms":"5000"

"heartbeat.interval.ms":"60000"

"session.timeout.ms":"300000

"max.poll.records":"100"