Error in simple Hadoop Map Reduce

Question

I tried running a hadoop map Reduce problem of word count. I set up HDFS and everything correctly to my knowledge.

When I execute it,

I get this error while trying to run a simple word count problem. I am new to Hadoop. Any help how to resolve this will be appreciated.

13/06/13 20:21:17 INFO input.FileInputFormat: Total input paths to process : 4
13/06/13 20:21:17 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/06/13 20:21:17 WARN snappy.LoadSnappy: Snappy native library not loaded
13/06/13 20:21:17 INFO mapred.JobClient: Running job: job_201306131940_0007
13/06/13 20:21:18 INFO mapred.JobClient:  map 0% reduce 0%
13/06/13 20:21:21 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stdout
13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stderr
13/06/13 20:21:24 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_1, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stdout
13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stderr
13/06/13 20:21:28 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_2, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stdout
13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stderr
13/06/13 20:21:34 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stdout
13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stderr
13/06/13 20:21:37 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_1, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stdout
13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stderr
13/06/13 20:21:40 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_2, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stdout
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stderr
13/06/13 20:21:43 INFO mapred.JobClient: Job complete: job_201306131940_0007
13/06/13 20:21:43 INFO mapred.JobClient: Counters: 4
13/06/13 20:21:43 INFO mapred.JobClient:   Job Counters 
13/06/13 20:21:43 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=24100
13/06/13 20:21:43 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/06/13 20:21:43 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/06/13 20:21:43 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0

Answer 1

As you are getting

java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)

It seems that the memory allocated for the tasks trackers is more than the nodes actual memory.

You can get it more here

Answer 2

I got the same exact error while trying to do your same thing - performing word count on simple text. In my case, I solved the issue changing the JDK version which Hadoop relies on. I'm using Hadoop 1.2.1, and I found online that the preferred Java version for it is 1.6