Clarification of some Hadoop Concepts

0 votes
I'm using hadoop to process a video using HVPI, an open-source interface. However, the implementation of the inputsplit, more precisely in the isSplitableobContext (context, Path file)method returns a false. By default, this method returns true but in the current implementation, there is a reason to return afalse. If this method returns false I will only have one map task. If I am not wrong, hadoop allocates for each input split a container that corresponds to the computational resources of a certain node of the network where a map task is executed and this node should preferably contain the data that will process. If I have a false I will only have an input split and consequently, just one map task and this map task will run only on a cluster node.   The big question is how an only map task take advantage of all the CPU resources of a cluster and not just a single container on a single node?
Sep 5, 2018 in Big Data Hadoop by Neha
• 6,300 points
467 views

1 answer to this question.

0 votes

ets try to understand what is the problem . 
1. One takes a file and divides it into fileSplits. 
2. Each split is consumed by one mapper. 
3. How do you make sure a record in the file is not split across two file splits. 
4. A record cant be ignored nor read partially. 
5. A InputFormat takes care of carefully splitting the file and handling situations when a record is split at the boundary of file splits. 
6. Hadoop has varios inpuit formats like TextInputFormat, KeyValueTextInputFormat

Try to find an input format which can be used for your video files or write one yourself. FileInputFormat is the base class for all.

Hope this answer helps :)

answered Sep 5, 2018 by Frankie
• 9,830 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What are some of the famous visualization tools which can be integrated with Hadoop & Hive?

I have personally used two visualization tools ...READ MORE

answered May 1, 2018 in Big Data Hadoop by coldcode
• 2,090 points
1,976 views
0 votes
1 answer

What do we exactly mean by “Hadoop” – the definition of Hadoop?

The official definition of Apache Hadoop given ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by Shubham
1,851 views
0 votes
1 answer

How to run example codes of Hadoop Definitive Guide book?

You will find multiple git repositories where ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 13,490 points
1,088 views
0 votes
1 answer

Types of Hadoop Clusters

Below is the difference given between Standalone, ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
3,754 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
11,002 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,515 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
108,656 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,596 views
0 votes
1 answer

Can I have a list of property files used in Hadoop Framework?

Here is a complete list of configuration ...READ MORE

answered Aug 14, 2018 in Big Data Hadoop by Frankie
• 9,830 points
841 views
0 votes
1 answer

Integration of Hadoop with Mongo DB concept

MongoDB isn't built to work on top ...READ MORE

answered Sep 25, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,742 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP