How to move data from Oracle database to Hadoop

Question

I've around 50 GB of data and my requirement is to import that data from Oracle database to Hadoop Distributed File System so that I can do further processing on that data.

So, can anyone tell me is there any way to load data from Oracle DB to HDFS?

I heard about a tool called Sqoop, but I don't have any idea how to use it?

Please help!

Thanks in advance.

Answer 1

Yes, you heard it correctly.

Apache Sqoop is one of the tool used for loading data from Oracle DB to HDFS. But, Sqoop is not just limited to just Oracle DB, you can use it to import and export data to & from most of the relational databases.

Let me tell you first about Sqoop.

Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and external data stores such as relational databases, enterprise data warehouses.

Sqoop is used to import data from external datastores into Hadoop Distributed File System or related Hadoop eco-systems like Hive and HBase. Similarly, Sqoop can also be used to extract data from Hadoop or its eco-systems and export it to external datastores such as relational databases, enterprise data warehouses. Sqoop works with relational databases such as Teradata, Netezza, Oracle, MySQL, Postgres etc.

You can refer the link: https://www.edureka.co/blog/hdfs-using-sqoop/

Follow the steps as mentioned in the blog and you will get to know how Sqoop is used to transfer data from Oracle DB to HDFS.

I hope this will answer your question to some extent.

Comments

how about the visa versa

transfer data from HDFS to Oracle?

do you have any practical example?

---

To transfer data from HDFS to Oracle you can use HIVE . This allows you to connect your database with HDFS.

You can go through the below link and end of the blog you will find similar kind of example.

https://www.edureka.co/blog/hive-commands-with-examples