Debezium Kafka connector mongodb : Error connecting kafka connector to mongodb

Question

Below are my MongoDB config in /etc/kafka/connect-mongodb-source.properties

name=mongodb-source-connector 
connector.class=io.debezium.connector.mongodb.MongoDbConnector 
mongodb.hosts=/remoteserveraddress:27017 
mongodb.name=mongo_conn 
initial.sync.max.threads=1 
tasks.max=1

but getting below error

 ERROR Plugin class loader for connector: 'io.debezium.connector.mongodb.MongoDbConnector' was not found. Returning: org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader@5a058be5 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:165)

Running connector in standalone mode.

I have mongodbconnector class under debezium-debezium-connector-mongodb-1.0.0/debezium-connector-mongodb-1.0.0.Final.jar

also classpath is set as follows

#for CLASSPATH
CLASSPATH=/Users/111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/*
export CLASSPATH
PATH=$PATH:/usr/local/sbin
export PATH

Using plugin path I see its able to register and load all required plugins..

[2020-01-10 08:14:07,916] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectStandalone:78)
[2020-01-10 08:14:07,942] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongodb-driver-3.11.1.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,082] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongodb-driver-3.11.1.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)
[2020-01-10 08:14:08,083] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,083] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,083] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,085] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/debezium-connector-mongodb-1.0.0.Final.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,120] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/debezium-connector-mongodb-1.0.0.Final.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)
[2020-01-10 08:14:08,121] INFO Added plugin 'io.debezium.connector.mongodb.MongoDbConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,121] INFO Added plugin 'io.debezium.connector.mongodb.transforms.ExtractNewDocumentState' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,121] INFO Added plugin 'io.debezium.connector.mongodb.transforms.UnwrapFromMongoDbEnvelope' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,122] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/debezium-core-1.0.0.Final.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,198] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/debezium-core-1.0.0.Final.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)
[2020-01-10 08:14:08,198] INFO Added plugin 'io.debezium.converters.ByteBufferConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,199] INFO Added plugin 'io.debezium.transforms.UnwrapFromEnvelope' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,199] INFO Added plugin 'io.debezium.transforms.ExtractNewRecordState' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,199] INFO Added plugin 'io.debezium.transforms.outbox.EventRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,199] INFO Added plugin 'io.debezium.transforms.ByLogicalTableRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,200] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongo-kafka-0.2-all.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,340] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongo-kafka-0.2-all.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)
[2020-01-10 08:14:08,340] INFO Added plugin 'com.mongodb.kafka.connect.MongoSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,340] INFO Added plugin 'com.mongodb.kafka.connect.MongoSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:193)
[2020-01-10 08:14:08,341] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/bson-3.11.1.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,373] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/bson-3.11.1.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)
[2020-01-10 08:14:08,373] INFO Loading plugin from: /Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongodb-driver-core-3.11.1.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:241)
[2020-01-10 08:14:08,465] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/1111111/workspace/KafkaConnect/confluent-5.3.2/debezium-debezium-connector-mongodb-1.0.0/mongodb-driver-core-3.11.1.jar} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:264)

but at the end again same issue

    [2020-01-10 08:40:43,613] ERROR Plugin class loader for connector: 'io.debezium.connector.mongodb.MongoDbConnector' was not found. Returning: org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader@33f2df51 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:165)
[2020-01-10 08:40:43,809] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:121)
java.lang.IllegalArgumentException: username can not be null
    at com.mongodb.MongoCredential.<init>(MongoCredential.java:350)
    at com.mongodb.MongoCredential.<init>(MongoCredential.java:344)
    at com.mongodb.MongoCredential.createCredential(MongoCredential.java:169)
    at io.debezium.connector.mongodb.ConnectionContext.<init>(ConnectionContext.java:69)
    at io.debezium.connector.mongodb.MongoDbConnector.validate(MongoDbConnector.java:222)
    at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:313)
    at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:192)
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:115)
[2020-01-10 08:40:43,810] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)

output of >curl localhost:8083/connector-plugins

[{"class":"com.mongodb.kafka.connect.MongoSinkConnector","type":"sink","version":"0.2"},{"class":"com.mongodb.kafka.connect.MongoSourceConnector","type":"source","version":"0.2"},{"class":"io.confluent.connect.activemq.ActiveMQSourceConnector","type":"source","version":"5.3.2"},{"class":"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector","type":"sink","version":"5.3.2"},{"class":"io.confluent.connect.ibm.mq.IbmMQSourceConnector","type":"source","version":"5.3.2"},{"class":"io.confluent.connect.jdbc.JdbcSinkConnector","type":"sink","version":"5.3.2"},{"class":"io.confluent.connect.jdbc.JdbcSourceConnector","type":"source","version":"5.3.2"},{"class":"io.confluent.connect.jms.JmsSourceConnector","type":"source","version":"5.3.2"},{"class":"io.confluent.connect.replicator.ReplicatorSourceConnector","type":"source","version":"5.3.2"},{"class":"io.confluent.connect.s3.S3SinkConnector","type":"sink","version":"5.3.2"},{"class":"io.confluent.connect.storage.tools.SchemaSourceConnector","type":"source","version":"5.3.2-ce"},{"class":"io.debezium.connector.mongodb.MongoDbConnector","type":"source","version":"1.0.0.Final"},{"class":"org.apache.kafka.connect.file.FileStr

thanks in advance

Answer 1

First of all, please check the installation of your plugin using the Kafka Connect REST Interface (see details here).

Try to install Kafka Connect plugins using the plugin path mechanism instead of CLASSPATH (more info in the docs).