Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

sqoop, select specific columns

Tags:

hadoop

sqoop

In the sqoop statements, is there a provision where we can select only specific columns from oracle side?

1 : works

sqoop import --target-dir /tmp/customers --query "SELECT * FROM schema1.customers where item>=1234 and \$CONDITIONS" --connect jdbc:oracle:thin:@server1.companyxyz.com:4567/prod --username xyz --password xyz --hive-drop-import-delims -m 8 --fields-terminated-by , --escaped-by \\ --split-by cust_id

2 : fails

sqoop import --target-dir /tmp/customers --query "SELECT cust_id, name, address, date, history, occupation FROM schema1.customers where item>=1234 and \$CONDITIONS" --connect jdbc:oracle:thin:@server1.companyxyz.com:4567/prod --username xyz --password xyz --hive-drop-import-delims -m 8 --fields-terminated-by , --escaped-by \\ --split-by cust_id
like image 873
sharp Avatar asked Oct 27 '25 21:10

sharp


1 Answers

you can use --columns --table --where clauses to achieve it. Sample is below:

sqoop import  
--connect jdbc:oracle:thin:@server1.companyxyz.com:4567/prod/DATABASE=schema1
--username xyz 
--password xyz 
--table customers
--columns cust_id, name, address, date, history, occupation  
--where item>=1234 
--target-dir /tmp//customers
--m 8
--split-by cust_id
--fields-terminated-by , 
--escaped-by \ 
--hive-drop-import-delims  
--map-column-java
  cust_id=string, name=string, address=string, date=string, history=string, occupation=string
like image 85
Ram Manohar Avatar answered Oct 31 '25 10:10

Ram Manohar



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!