May I use Apache NIFI for transfering files to HDFS system. [closed]

Question

If yes please guide me basic steps for this. I scorll through pages and realized that its possible to do it using core-site.xml

Answer 1

You can easily transferring files to HDFS File System by using PutHDFS Processor.

Look at this below Link.,

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.PutHDFS/

In this processor you have to specify following properties is enough to transfer files into HDFS.

1.Hadoop Configuration Resources: -A file or comma separated list of files

/Hadoop/conf/core-site.xml,/Hadoop/conf/hdfs-site.xml,/Hadoop/conf/yarn-site.xml,/Hadoop/conf/mapred-site.xml

2.Directory: Specify location of your HDFS Directory

/FilesFromNiFi/

3.Conflict Strategy:(append/replace/ignore/fail)

append

For example:

If u need to move the file from local drive into HDFS means using below NiFi workflow

GetFile-->PutHDFS

GetFile-->Just give your input directory.

putHDFS-->Specify your HDFS location 

You need to drag a processor into UI with respect to below image.,

Refer this link to drag this two processors GetFile and putHDFS into NiFi UI

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#adding-components-to-the-canvas

http://zdatainc.com/2016/02/hello-nifi-data-orchestration-using-hortonworks-dataflow-hdf/

let me know if you face any problem.