Update hdfs data before stroring in MySql

0 votes
Hi guys. I am new to Hadoop and I have some doubt. I am learning hdfs and Sqoop. I created a table in SQL and then imported it to hdfs using Sqoop. Now I want to store the data in SQL from hdfs again. But the problem is that I want to make some changes to the data before storing it in SQL. How can I do this? Is intermediate data updation possible?
Jan 27 in Big Data Hadoop by Shubhash
41 views

1 answer to this question.

0 votes

Yes, you can update the data before storing it in SQL table. In this approach, you will have to update the column using hive and store the hive output to hdfs using INSERT OVERWRITE DIRECTORY. And after that, to store it in SQL, you can use the Sqoop command. Example:

sqoop export \
--connect jdbc:mysql://master/poc \
--username root \
--table employee \
--export-dir /user/hdfs/mysql/export.txt \
--update-key id \
--update-mode allowinsert \
--fields-terminated-by '\t' \
-m 1
answered Jan 27 by Omkar
• 68,480 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Is there any way to load data from MySql to HDFS?

The generic command i.e used to import ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
727 views
0 votes
1 answer

Synchronizing sqoop imported data in HDFS

Data can be synchronised using incremental parameter ...READ MORE

answered Jul 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
158 views
0 votes
1 answer

How can I change data in HDFS?

You can not modified data once stored ...READ MORE

answered Feb 18 in Big Data Hadoop by Rishi
97 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
3,580 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
455 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,447 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,334 views
0 votes
1 answer

How to send curl downloaded data in hdfs?

Try using put command with stdin as ...READ MORE

answered Jan 25 in Big Data Hadoop by Omkar
• 68,480 points
97 views
0 votes
1 answer

Update new sql data in sqoop

You can use incremental for this. When ...READ MORE

answered Feb 11 in Big Data Hadoop by Omkar
• 68,480 points
113 views