Most voted questions in Big Data Hadoop

0 votes
1 answer

What does AM in “AM Container” mean in Hadoop

Here's another link from Hadoop which may ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,160 views
0 votes
1 answer

Hadoop: How to keep duplicates in Hive using collect_set()?

SELECT hash_id, COLLECT_LIST(num_of_cats) AS ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 2,058 views
0 votes
1 answer

Hadoop: `.' no such file or directory while installing

Try hadoop fs -mkdir -p /user/[Username] and then run ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,341 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 18,687 views
0 votes
1 answer

Job and Task Scheduling In Hadoop

I  n case of fair scheduler, when there is ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 1,639 views
0 votes
1 answer

Hadoop: How to Group mongodb - mapReduce output?

db.order.mapReduce(function() { emit (this.customer,{count:1,orderDate:this.orderDate.interval_start}) }, function(key,values){ var category; ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 679 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Mark
 • 16,580 views
0 votes
1 answer

Hadoop Mapreduce: Error: Could not find or load main class com.sun.tools.javac.Main

You have to add HADOOP_CLASSPATH environment parameter: expor ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 3,251 views
0 votes
1 answer

How to delete hdfs folder from java app?

Try this: Configuration conf = new Configuration(); ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,892 views
0 votes
1 answer

Hadoop Mapreduce: Class Not Found Exception

Try adding this Job job = new Job(conf, ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 3,558 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

Using PySpark  hadoop = sc._jvm.org.apache.hadoop fs = hadoop.fs.FileSystem conf = ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Kiran
 • 10,577 views
0 votes
1 answer

Opening a HDFS file in browser

In HDFS 8020 is IPC port, you ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 9,306 views
0 votes
1 answer

How to uninstall all versions of hadoop completely from the system?

Do a: sudo dpkg -l | grep hadoop to ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 10,095 views
0 votes
1 answer

Understanding LongWritable

Hadoop needs to be able to serialise ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,135 views
0 votes
1 answer

How do I compile my java program on Ubuntu such that it will refer to hadoop-2.2.0 libraries?

The simplest solution for Linux machines would ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 896 views
0 votes
1 answer

Hadoop: No Such Method Exception

Mapper and reducer classes need to be ...READ MORE

Oct 26, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,647 views
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

Aug 1, 2019 in Big Data Hadoop by Dinish
 • 17,380 views
0 votes
1 answer

How to fix corrupt HDFS FIles?

You can use hdfs fsck / to ...READ MORE

Oct 25, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,849 views
0 votes
3 answers

How to specify KeyValueTextInputFormat Separator in Hadoop-.20 api?

conf.set("key.value.separator.in.input.line", ","); Job job = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Rio
 • 1,576 views
0 votes
1 answer

Where is hadoop-test.jar in Hadoop2?

In Hadoop 1, testmapredsort is launched from AllTestDriver.java, which was ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 879 views
0 votes
1 answer

What is Hadoop Performance Tuning?

Is it consistent or one-off reading? Is ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 765 views
0 votes
1 answer

Google Hadoop Filesystem Encryption

If you're asking if customer-supplied encryption keys ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 515 views
0 votes
1 answer

Is Java alone Sufficient for Hadoop?

In my day job, I've just spent ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 450 views
0 votes
1 answer

What is -cp command in hadoop? How it works?

/user/cloudera/data1 is not a directory, it is ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 3,845 views
0 votes
1 answer

What is a data serialization system?

So when Hadoop was being written by ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,049 views
0 votes
1 answer

What is the best way to integrate SAS with Hadoop without losing the parallel processing capacity of Hadoop

One of the major pushes at SAS ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,962 views
0 votes
1 answer

How to configure Yarn from command line?

It's a bit non-intuitive, but it turns ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,658 views
0 votes
1 answer

Does Hadoop and Spark support iPv6 now?

It does not. As of today HADOOP-11890, which ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,752 views
0 votes
1 answer

Hadoop Cluster Node Setup.

Yes, at least in small clusters those ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 426 views
0 votes
1 answer

What are the prerequisites to learn Hadoop in java perspective?

In my day job, I've just spent ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 601 views
0 votes
1 answer

POC for Hadoop in real time scenario

I'd like to point a few things. If ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,459 views
0 votes
1 answer

PIG - Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Yes, it is a compatibility issue. in Hadoop ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 924 views
0 votes
1 answer

Setting textinputformat.record.delimiter in spark

I got this working with plain uncompressed ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 2,145 views
0 votes
1 answer

Is it possible to integrate Kdb+ and hadoop?

kdb is a column oriented database plus ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 778 views
0 votes
1 answer

Is MapR a substitute for MapReduce?

MapR is a commercial distribution of Apache ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 960 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 506 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 16,544 views
0 votes
2 answers

Sqoop: Could not load mysql driver exception.

I add mysql driver for sqoop and it ...READ MORE

Mar 1, 2020 in Big Data Hadoop by Manh Quang
 • 180 points • 3,336 views
0 votes
1 answer

Hadoop on OSX “Unable to load realm info from SCDynamicStore”

Add the following to your hadoop-env.sh file: export ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 510 views
0 votes
1 answer

How can I calculate exact median with Apache Spark?

You need to sort RDD and take ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 4,101 views
0 votes
1 answer

How can I access S3/S3n from a local Hadoop 2.6 installation?

For some reason, the jar hadoop-aws-[version].jar which contains the ...READ MORE

Oct 3, 2018 in Big Data Hadoop by digger
 • 26,740 points • 5,104 views
0 votes
1 answer

What is Hadoop Speculative task execution?

One problem with the Hadoop system is ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 883 views
0 votes
1 answer

What is the difference between Writable & WritableComparable in Hadoop?

Writable in an interface in Hadoop and types ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 7,284 views
0 votes
1 answer

out of Memory Error in Hadoop

You can assign more memory by editing ...READ MORE

Oct 1, 2018 in Big Data Hadoop by slayer
 • 29,350 points • 1,463 views
0 votes
1 answer

Iterate twice on values (MapReduce)

We have to cache the values from ...READ MORE

Oct 1, 2018 in Big Data Hadoop by digger
 • 26,740 points • 1,904 views
0 votes
2 answers

How to convert .txt file to Hadoop's sequence file format

import java.io.IOException; import java.net.URI; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IOUtils; import ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Sanjay
 • 3,186 views
0 votes
1 answer

Using Hadoop for Data Analytics.

Yes, your approach is correct - you ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 521 views
0 votes
1 answer

How to write a file in HDFS using Java Programming language?

Define the HADOOP_CONF_DIR environment variable to your Hadoop configuration ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,630 views
0 votes
1 answer

What should be the choice of database and what type of data format is suitable for Spark/hadoop?

Use Parquet. I'm not sure about CSV ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 776 views
0 votes
1 answer

When I try to connect to Hadoop, I get a error as -"Connecting to Resource Manager failed".

Make sure you've started Yarn. Use this ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,154 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.