Michael Heil questions

Michael Heil answers

• How to distribute messages between Kafka topics with different configuration?
• Calculating a moving average column using pyspark structured streaming
• Delete a specific record in a Kafka topic using compaction
• Consuming kafka batch for multiple partitions
• Read Kafka Topic from beginning
• Can Kafka be used as a distribute work queue
• How to use transactions in Kafka and how to use abortTransaction?
• When does Kafka rotates partition leader? Do we have any property to control it?
• Unable to overwrite default value of "spark.sql.shuffle.partitions" with Spark Structured Streaming
• Kafka Producer guarantees