Hadoop regarding input split

0 votes
How does the input split work? What is the process?
Dec 27, 2018 in Big Data Hadoop by slayer
• 29,040 points
13 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
+1 vote

Hadoop framework divides a large file into blocks (64MB or 128 MB) and stores these blocks in the slave nodes. HDFS is unaware of the content of the block. Suppose the record crosses the block limit, then a part of the same record is written on one block and the other is written on another block.

Hadoop tracks this split of data by the logical representation of the data known as Input Split. When Map Reduce client calculates the input splits, it actually checks if the entire record resides in the same block or not. If the record overheads and some part of it is written into another block, the input split captures the location information of the next Block and byte offset of the data are needed to complete the record.

answered Dec 27, 2018 by Omkar
• 65,850 points

Related Questions In Big Data Hadoop

0 votes
2 answers

Hadoop query regarding setJarByClass method of Job class

http://www.bigdataspeak.com/2014/06/what-is-need-to-use-jobsetjarbycla ...READ MORE

answered Feb 6 in Big Data Hadoop by siri
291 views
0 votes
2 answers

Hadoop error: Input path does not exist

I was able to fix the issue. ...READ MORE

answered Jan 12 in Big Data Hadoop by Ponezhil Ramu
403 views
0 votes
1 answer

How to set input split settings?

It can be controlled by setting the ...READ MORE

answered Dec 27, 2018 in Big Data Hadoop by Omkar
• 65,850 points
44 views
0 votes
1 answer

How can Hadoop process the records that are split across the block boundaries?

First of all, Map Reduce algorithm is not programmed ...READ MORE

answered Apr 15 in Big Data Hadoop by nitinrawat895
• 9,070 points
55 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,070 points
1,679 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,070 points
132 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
8,164 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
573 views
0 votes
1 answer

Hadoop Hive: How to split string in Hive?

You can use the split function along ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 65,850 points
786 views
0 votes
1 answer

Hadoop Hive: How to split a single row into multiple rows?

Try this SELECT ID1, Sub FROM tableName lateral view ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 65,850 points
443 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.