How do we map multiple collections to multiple topics in the mongodb-sink-connector?

Question

I am using a mongodb sink connector to read from some topics json data, and then write them into their respective collections in mongodb. But, when I give multiple collections with topic-collection mappings, the connector doesn't recognize the collection names, and instead create collections with the name same as the topics'. Below is the worker config:

name=sink-mongonew_Transaction_test4
connector.class=com.mongodb.kafka.connect.MongoSinkConnector
connection.uri=mongodb://****:27017
database=ec_ods
topics=t6,t5
collections=abc,def
collection.t6=abc
collection.t5=def
type.name=kafka-connect
key.ignore=true
document.id.strategy.abc=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy
value.projection.type.abc=whitelist
value.projection.list.abc=ID
writemodel.strategy.abc=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneTimestampsStrategy
document.id.strategy.def=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy
value.projection.type.def=whitelist
value.projection.list.def=ID
writemodel.strategy.def=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneTimestampsStrategy
delete.on.null.values=false
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false

Answer 1

Here a link to a proper documentation of the MongoDB Kafka sink connector https://github.com/mongodb-labs/mongo-kafka/blob/master/docs/sink.md

As per documentation, you should use the topic.override.%s.%s property

The overrides configuration allows for per topic customization of configuration. The customized overrides are merged with the default configuration, to create the specific configuration for a topic. For example, topic.override.foo.collection=bar will store data from the foo topic into the bar collection. Note: All configuration options apart from 'connection.uri' and 'topics' are overridable. string ""