Update hdfs data before stroring in MySql

0 votes
Hi guys. I am new to Hadoop and I have some doubt. I am learning hdfs and Sqoop. I created a table in SQL and then imported it to hdfs using Sqoop. Now I want to store the data in SQL from hdfs again. But the problem is that I want to make some changes to the data before storing it in SQL. How can I do this? Is intermediate data updation possible?
Jan 27, 2019 in Big Data Hadoop by Shubhash
47 views

1 answer to this question.

0 votes

Yes, you can update the data before storing it in SQL table. In this approach, you will have to update the column using hive and store the hive output to hdfs using INSERT OVERWRITE DIRECTORY. And after that, to store it in SQL, you can use the Sqoop command. Example:

sqoop export \
--connect jdbc:mysql://master/poc \
--username root \
--table employee \
--export-dir /user/hdfs/mysql/export.txt \
--update-key id \
--update-mode allowinsert \
--fields-terminated-by '\t' \
-m 1
answered Jan 27, 2019 by Omkar
• 68,860 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Is there any way to load data from MySql to HDFS?

The generic command i.e used to import ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,840 points
852 views
0 votes
1 answer

Synchronizing sqoop imported data in HDFS

Data can be synchronised using incremental parameter ...READ MORE

answered Jul 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,290 points
176 views
0 votes
1 answer

How can I change data in HDFS?

You can not modified data once stored ...READ MORE

answered Feb 18, 2019 in Big Data Hadoop by Rishi
112 views
0 votes
1 answer

How does Hadoop process data which is split across multiple boundaries in an HDFS?

I found some comments: from the Hadoop ...READ MORE

answered Jul 1, 2019 in Big Data Hadoop by ravikiran
• 4,600 points
53 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
3,915 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
539 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
20,820 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,290 points
1,476 views
0 votes
1 answer

How to send curl downloaded data in hdfs?

Try using put command with stdin as ...READ MORE

answered Jan 25, 2019 in Big Data Hadoop by Omkar
• 68,860 points
115 views
0 votes
1 answer

Update new sql data in sqoop

You can use incremental for this. When ...READ MORE

answered Feb 11, 2019 in Big Data Hadoop by Omkar
• 68,860 points
145 views