Most voted questions in Big Data Hadoop

0 votes
1 answer

What does AM in “AM Container” mean in Hadoop

Here's another link from Hadoop which may ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,174 views
0 votes
1 answer

Hadoop: How to keep duplicates in Hive using collect_set()?

SELECT hash_id, COLLECT_LIST(num_of_cats) AS ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,081 views
0 votes
1 answer

Hadoop: `.' no such file or directory while installing

Try hadoop fs -mkdir -p /user/[Username] and then run ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,356 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 18,758 views
0 votes
1 answer

Job and Task Scheduling In Hadoop

I  n case of fair scheduler, when there is ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 1,650 views
0 votes
1 answer

Hadoop: How to Group mongodb - mapReduce output?

db.order.mapReduce(function() { emit (this.customer,{count:1,orderDate:this.orderDate.interval_start}) }, function(key,values){ var category; ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 688 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Mark
 • 16,628 views
0 votes
1 answer

Hadoop Mapreduce: Error: Could not find or load main class com.sun.tools.javac.Main

You have to add HADOOP_CLASSPATH environment parameter: expor ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,281 views
0 votes
1 answer

How to delete hdfs folder from java app?

Try this: Configuration conf = new Configuration(); ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,909 views
0 votes
1 answer

Hadoop Mapreduce: Class Not Found Exception

Try adding this Job job = new Job(conf, ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,586 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

Using PySpark  hadoop = sc._jvm.org.apache.hadoop fs = hadoop.fs.FileSystem conf = ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Kiran
 • 10,609 views
0 votes
1 answer

Opening a HDFS file in browser

In HDFS 8020 is IPC port, you ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 9,348 views
0 votes
1 answer

How to uninstall all versions of hadoop completely from the system?

Do a: sudo dpkg -l | grep hadoop to ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 10,133 views
0 votes
1 answer

Understanding LongWritable

Hadoop needs to be able to serialise ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,157 views
0 votes
1 answer

How do I compile my java program on Ubuntu such that it will refer to hadoop-2.2.0 libraries?

The simplest solution for Linux machines would ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 914 views
0 votes
1 answer

Hadoop: No Such Method Exception

Mapper and reducer classes need to be ...READ MORE

Oct 26, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,660 views
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

Aug 1, 2019 in Big Data Hadoop by Dinish
 • 17,435 views
0 votes
1 answer

How to fix corrupt HDFS FIles?

You can use hdfs fsck / to ...READ MORE

Oct 25, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,874 views
0 votes
3 answers

How to specify KeyValueTextInputFormat Separator in Hadoop-.20 api?

conf.set("key.value.separator.in.input.line", ","); Job job = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Rio
 • 1,599 views
0 votes
1 answer

Where is hadoop-test.jar in Hadoop2?

In Hadoop 1, testmapredsort is launched from AllTestDriver.java, which was ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 894 views
0 votes
1 answer

What is Hadoop Performance Tuning?

Is it consistent or one-off reading? Is ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 773 views
0 votes
1 answer

Google Hadoop Filesystem Encryption

If you're asking if customer-supplied encryption keys ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 526 views
0 votes
1 answer

Is Java alone Sufficient for Hadoop?

In my day job, I've just spent ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 457 views
0 votes
1 answer

What is -cp command in hadoop? How it works?

/user/cloudera/data1 is not a directory, it is ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 3,866 views
0 votes
1 answer

What is a data serialization system?

So when Hadoop was being written by ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,055 views
0 votes
1 answer

What is the best way to integrate SAS with Hadoop without losing the parallel processing capacity of Hadoop

One of the major pushes at SAS ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,977 views
0 votes
1 answer

How to configure Yarn from command line?

It's a bit non-intuitive, but it turns ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,671 views
0 votes
1 answer

Does Hadoop and Spark support iPv6 now?

It does not. As of today HADOOP-11890, which ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,766 views
0 votes
1 answer

Hadoop Cluster Node Setup.

Yes, at least in small clusters those ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 448 views
0 votes
1 answer

What are the prerequisites to learn Hadoop in java perspective?

In my day job, I've just spent ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 617 views
0 votes
1 answer

POC for Hadoop in real time scenario

I'd like to point a few things. If ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,466 views
0 votes
1 answer

PIG - Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Yes, it is a compatibility issue. in Hadoop ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 933 views
0 votes
1 answer

Setting textinputformat.record.delimiter in spark

I got this working with plain uncompressed ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,169 views
0 votes
1 answer

Is it possible to integrate Kdb+ and hadoop?

kdb is a column oriented database plus ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 787 views
0 votes
1 answer

Is MapR a substitute for MapReduce?

MapR is a commercial distribution of Apache ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 975 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 513 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 16,580 views
0 votes
2 answers

Sqoop: Could not load mysql driver exception.

I add mysql driver for sqoop and it ...READ MORE

Mar 1, 2020 in Big Data Hadoop by Manh Quang
 • 180 points • 3,355 views
0 votes
1 answer

Hadoop on OSX “Unable to load realm info from SCDynamicStore”

Add the following to your hadoop-env.sh file: export ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 524 views
0 votes
1 answer

How can I calculate exact median with Apache Spark?

You need to sort RDD and take ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,114 views
0 votes
1 answer

How can I access S3/S3n from a local Hadoop 2.6 installation?

For some reason, the jar hadoop-aws-[version].jar which contains the ...READ MORE

Oct 3, 2018 in Big Data Hadoop by digger
 • 26,740 points • 5,119 views
0 votes
1 answer

What is Hadoop Speculative task execution?

One problem with the Hadoop system is ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 893 views
0 votes
1 answer

What is the difference between Writable & WritableComparable in Hadoop?

Writable in an interface in Hadoop and types ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 7,421 views
0 votes
1 answer

out of Memory Error in Hadoop

You can assign more memory by editing ...READ MORE

Oct 1, 2018 in Big Data Hadoop by slayer
 • 29,350 points • 1,471 views
0 votes
1 answer

Iterate twice on values (MapReduce)

We have to cache the values from ...READ MORE

Oct 1, 2018 in Big Data Hadoop by digger
 • 26,740 points • 1,910 views
0 votes
2 answers

How to convert .txt file to Hadoop's sequence file format

import java.io.IOException; import java.net.URI; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IOUtils; import ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Sanjay
 • 3,217 views
0 votes
1 answer

Using Hadoop for Data Analytics.

Yes, your approach is correct - you ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 530 views
0 votes
1 answer

How to write a file in HDFS using Java Programming language?

Define the HADOOP_CONF_DIR environment variable to your Hadoop configuration ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,641 views
0 votes
1 answer

What should be the choice of database and what type of data format is suitable for Spark/hadoop?

Use Parquet. I'm not sure about CSV ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 785 views
0 votes
1 answer

When I try to connect to Hadoop, I get a error as -"Connecting to Resource Manager failed".

Make sure you've started Yarn. Use this ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,164 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.