Update hdfs data before stroring in MySql

0 votes
Hi guys. I am new to Hadoop and I have some doubt. I am learning hdfs and Sqoop. I created a table in SQL and then imported it to hdfs using Sqoop. Now I want to store the data in SQL from hdfs again. But the problem is that I want to make some changes to the data before storing it in SQL. How can I do this? Is intermediate data updation possible?
Jan 27, 2019 in Big Data Hadoop by Shubhash
51 views

1 answer to this question.

0 votes

Yes, you can update the data before storing it in SQL table. In this approach, you will have to update the column using hive and store the hive output to hdfs using INSERT OVERWRITE DIRECTORY. And after that, to store it in SQL, you can use the Sqoop command. Example:

sqoop export \
--connect jdbc:mysql://master/poc \
--username root \
--table employee \
--export-dir /user/hdfs/mysql/export.txt \
--update-key id \
--update-mode allowinsert \
--fields-terminated-by '\t' \
-m 1
answered Jan 27, 2019 by Omkar
• 68,940 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Is there any way to load data from MySql to HDFS?

The generic command i.e used to import ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,840 points
930 views
0 votes
1 answer

Synchronizing sqoop imported data in HDFS

Data can be synchronised using incremental parameter ...READ MORE

answered Jul 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
190 views
0 votes
1 answer

How can I change data in HDFS?

You can not modified data once stored ...READ MORE

answered Feb 18, 2019 in Big Data Hadoop by Rishi
123 views
0 votes
1 answer

How does Hadoop process data which is split across multiple boundaries in an HDFS?

I found some comments: from the Hadoop ...READ MORE

answered Jul 1, 2019 in Big Data Hadoop by ravikiran
• 4,600 points
59 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
4,147 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
586 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
22,414 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,552 views
0 votes
1 answer

How to send curl downloaded data in hdfs?

Try using put command with stdin as ...READ MORE

answered Jan 25, 2019 in Big Data Hadoop by Omkar
• 68,940 points
131 views
0 votes
1 answer

Update new sql data in sqoop

You can use incremental for this. When ...READ MORE

answered Feb 11, 2019 in Big Data Hadoop by Omkar
• 68,940 points
163 views