SQOOP export , want to move million records to netezza from hive , guidance required

0 votes
Hi,

We were trying to export the data using below command, the speed at which it gets records is 2000 records every 15 minutes

Kindly do provide us few parameters which speeds up the process

sqoop --connect "jdbc:netezza://server/db?schema=schema" --table nz_table --username nz_usr -P --hcatalog-database hdw_db --hcatalog-table hdp_tbl --num-mappers 20 ;

Thanks

Subramanian
Apr 11 by Subramanian
• 120 points
223 views

1 answer to this question.

0 votes

Hi@Subramanian,

You can use the following parameter in your code.

The property sqoop.export.records.per.statement to specify the number of records that will be used in each insert statement.

Also, you can set how many rows will be inserted per transaction with the sqoop.export.statements.per.transaction property.

Use this parameters and try once.

answered Apr 13 by MD
• 55,020 points

Related Questions

0 votes
1 answer

How to transfer data from Netezza to HDFS using Apache Sqoop?

Remove the --direct option. It gives issue ...READ MORE

answered Apr 23, 2018 in Big Data Hadoop by kurt_cobain
• 9,320 points
663 views
0 votes
1 answer
0 votes
1 answer

Is there any way to load data from MySql to HDFS?

The generic command i.e used to import ...READ MORE

answered Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,950 points
1,490 views
0 votes
1 answer

How to move data from Oracle database to Hadoop?

Yes, you heard it correctly. Apache Sqoop is ...READ MORE

answered Apr 11, 2018 in Big Data Hadoop by nitinrawat895
• 10,950 points
3,740 views
0 votes
1 answer

Unable to move Hadoop Hive tables from Source to Destination.

First, check the permissions of HDFS Directory ...READ MORE

answered Nov 30, 2018 in Big Data Hadoop by Omkar
• 69,030 points
2,129 views
0 votes
1 answer

How to export file from sqoop to sql table?

For SQOOP export please try below command: bin/sqoop ...READ MORE

answered Dec 14, 2018 in Big Data Hadoop by Omkar
• 69,030 points
249 views
0 votes
1 answer

How to import tables into Hive from Sqoop?

Very simple, in your sqoop command that ...READ MORE

answered May 15, 2019 in Big Data Hadoop by John
235 views
0 votes
1 answer

How to remove duplicate records from Hive table?

A record is duplicate if there are ...READ MORE

answered Jul 11, 2019 in Big Data Hadoop by Bhuvan
3,480 views
0 votes
1 answer

Sqoop - moving data from DB2 to HDFS ( hive partition table)

sqoop import --driver com.ibm.db2.jcc.DB2Driver --connect jdbc:db2://db2.my.com:50000/databaseName --username ...READ MORE

answered Aug 9, 2019 in Big Data Hadoop by Payal
1,494 views
0 votes
0 answers

I want to download 2k excel records and rename the excel based on the column names from the table

I want to download these excels one ...READ MORE

Feb 17 in RPA by Hridya
• 120 points
176 views