Why we use --split by command in Sqoop?

0 votes

Why we use --split-by command in Sqoop?

Apr 11, 2019 in Big Data Hadoop by rashmi
3,232 views

2 answers to this question.

0 votes

The command --split-by is used to specify the column of the table used to generate splits for imports. This means that it specifies which column will be used to create the split while importing the data into the cluster.

Basically it is used to improve the import performance to achieve faster parallelism.

answered Apr 11, 2019 by Gitika
• 29,090 points
+2 votes
In simple explanation,

When specify SPLIT_BY only for sqoop'ing the whole table [which would be select * from table]. As this not condition based we need a logic to divide the data and process them in more than one node.

So we select a column based on which data can be divided base on a range [for integers] & characters [alphabets], to avoid huge amount of the data being concentrated on one mapper/node.

Bigdata is specifically meant to achieve parallelism.
answered Feb 6 by Ramji Sridaran

Related Questions In Big Data Hadoop

0 votes
1 answer

Why we use 'help' command in Hadoop Sqoop?

Hi, The command sqoop help lists the tools ...READ MORE

answered Feb 4 in Big Data Hadoop by MD
• 21,340 points
80 views
+1 vote
1 answer

Why do we use STORE command in pig?

Hey, We use store command to store the ...READ MORE

answered May 7, 2019 in Big Data Hadoop by Gitika
• 29,090 points
172 views
0 votes
1 answer

Why should we use "distinct" keyword in pig script?

Hey, The "distinct" statement is very simple. It ...READ MORE

answered May 3, 2019 in Big Data Hadoop by Gitika
• 29,090 points
84 views
0 votes
1 answer

Why we use Relation keyword in pig?

Hey, In pig, Relation represents a complete database. ...READ MORE

answered May 7, 2019 in Big Data Hadoop by Gitika
• 29,090 points
63 views
0 votes
1 answer
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
409 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
5,043 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,920 points
718 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
29,690 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,883 views