Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Hadoop: How to generate custom reduce output file name?

Tags:

hadoop

Now, I use MultipuleOuputs. I would like to remove the suffix string "-00001" from reducer's output filename such as "xxxx-[r/m]-00001".

Is there any idea?

Thanks.

like image 493
user3304477 Avatar asked Jan 24 '26 13:01

user3304477


1 Answers

From Hadoop javadoc to the write() method of MultipleOutputs:

Output path is a unique file generated for the namedOutput. For example, {namedOutput}-(m|r)-{part-number}

So you need to rename or merge these files on the HDFS.

like image 79
luish Avatar answered Jan 26 '26 05:01

luish



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!