Update hdfs data before stroring in MySql

0 votes
Hi guys. I am new to Hadoop and I have some doubt. I am learning hdfs and Sqoop. I created a table in SQL and then imported it to hdfs using Sqoop. Now I want to store the data in SQL from hdfs again. But the problem is that I want to make some changes to the data before storing it in SQL. How can I do this? Is intermediate data updation possible?
Jan 27 in Big Data Hadoop by Shubhash
18 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Yes, you can update the data before storing it in SQL table. In this approach, you will have to update the column using hive and store the hive output to hdfs using INSERT OVERWRITE DIRECTORY. And after that, to store it in SQL, you can use the Sqoop command. Example:

sqoop export \
--connect jdbc:mysql://master/poc \
--username root \
--table employee \
--export-dir /user/hdfs/mysql/export.txt \
--update-key id \
--update-mode allowinsert \
--fields-terminated-by '\t' \
-m 1
answered Jan 27 by Omkar
• 66,880 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Is there any way to load data from MySql to HDFS?

The generic command i.e used to import ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 9,390 points
348 views
0 votes
1 answer

Synchronizing sqoop imported data in HDFS

Data can be synchronised using incremental parameter ...READ MORE

answered Jul 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
57 views
0 votes
1 answer

How can I change data in HDFS?

You can not modified data once stored ...READ MORE

answered Feb 18 in Big Data Hadoop by Rishi
47 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,390 points
1,837 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,390 points
158 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
9,083 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
652 views
0 votes
1 answer

How to send curl downloaded data in hdfs?

Try using put command with stdin as ...READ MORE

answered Jan 25 in Big Data Hadoop by Omkar
• 66,880 points
15 views
0 votes
1 answer

Update new sql data in sqoop

You can use incremental for this. When ...READ MORE

answered Feb 11 in Big Data Hadoop by Omkar
• 66,880 points
29 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.