Hive: Whenever it fires a map reduce it gives me this error "Can not create a Path from an empty string", how do I debug?

Question

I am using hive 0.10 and when I do

hive -e "show tables", hive -e "desc table_name" it works!

but when I do something like hive -e "select count(*) table_name I am getting the exception below. Is there a way I can debug this? The same code worked in the previous cluster with a older version of hive and the new cluster is throwing this error. What should be the right way to debug this kind of issues, did not find anything from google that solves the issue.

    java.lang.IllegalArgumentException: Can not create a Path from an empty string
    at org.apache.hadoop.fs.Path.checkPathArg(Path.java:91)
    at org.apache.hadoop.fs.Path.<init>(Path.java:99)
    at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:382)
    at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:195)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:472)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1352)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1138)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

Answer 1

I have solved the problem.

I looked up the log file and in my case the table is an external table referring to a directory located on hdfs. This directory contains more than 300000 files. So while reading the files it was throwing an out of memory exception and may be for this reason it was getting an empty string and throwing 'Can not create a Path from an empty string' exception.

I tried with a smaller subset of files and it worked.

Bottomline, one possible cause for this exception is running out of memory.