How to configure hadoop to use non-default port: "0.0.0.0: ssh: connect to host 0.0.0.0 port 22: Connection refused"

Question

When I run start-dfs I get the below error and it looks like I need to tell hadoop to use a different port since that is what I require when I ssh into localhost. In other words the following works successfully: ssh -p 2020 localhost.

[Wed Jan 06 16:57:34 root@~]# start-dfs.sh
16/01/06 16:57:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: namenode running as process 85236. Stop it first.
localhost: datanode running as process 85397. Stop it first.
Starting secondary namenodes [0.0.0.0]
0.0.0.0: ssh: connect to host 0.0.0.0 port 22: Connection refused
16/01/06 16:57:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

core-site.xml:

<configuration>
    <property>
        <name>fs.default.name</name>
            <value>hdfs://localhost:9000</value>
    </property>
</configuration>

hdfs-site.xml:

<configuration>
    <property>
        <name>dfs.replication</name>
            <value>1</value>
    </property>

    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///hadoop/hdfs/namenode</value>
    </property>

    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:///hadoop/hdfs/datanode</value>
    </property>
</configuration>

Answer 1

If your Hadoop cluster nodes run sshd listening on a non-standard port, then it is possible to tell the Hadoop scripts to initiate ssh connections to that port. In fact, it's possible to customize any of the options passed to the ssh command.

This is controlled by an environment variable named HADOOP_SSH_OPTS. You can edit your hadoop-env.sh file and define it there. (By default this environment variable is not defined.)

For example:

export HADOOP_SSH_OPTS="-p 2020"