Most voted questions in Big Data Hadoop

0 votes
1 answer

What does AM in “AM Container” mean in Hadoop

Here's another link from Hadoop which may ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,166 views
0 votes
1 answer

Hadoop: How to keep duplicates in Hive using collect_set()?

SELECT hash_id, COLLECT_LIST(num_of_cats) AS ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 2,069 views
0 votes
1 answer

Hadoop: `.' no such file or directory while installing

Try hadoop fs -mkdir -p /user/[Username] and then run ...READ MORE

Nov 2, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,348 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 18,726 views
0 votes
1 answer

Job and Task Scheduling In Hadoop

I  n case of fair scheduler, when there is ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 1,646 views
0 votes
1 answer

Hadoop: How to Group mongodb - mapReduce output?

db.order.mapReduce(function() { emit (this.customer,{count:1,orderDate:this.orderDate.interval_start}) }, function(key,values){ var category; ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 684 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Mark
 • 16,602 views
0 votes
1 answer

Hadoop Mapreduce: Error: Could not find or load main class com.sun.tools.javac.Main

You have to add HADOOP_CLASSPATH environment parameter: expor ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 3,267 views
0 votes
1 answer

How to delete hdfs folder from java app?

Try this: Configuration conf = new Configuration(); ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,901 views
0 votes
1 answer

Hadoop Mapreduce: Class Not Found Exception

Try adding this Job job = new Job(conf, ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 3,576 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

Using PySpark  hadoop = sc._jvm.org.apache.hadoop fs = hadoop.fs.FileSystem conf = ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Kiran
 • 10,592 views
0 votes
1 answer

Opening a HDFS file in browser

In HDFS 8020 is IPC port, you ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 9,331 views
0 votes
1 answer

How to uninstall all versions of hadoop completely from the system?

Do a: sudo dpkg -l | grep hadoop to ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 10,111 views
0 votes
1 answer

Understanding LongWritable

Hadoop needs to be able to serialise ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,144 views
0 votes
1 answer

How do I compile my java program on Ubuntu such that it will refer to hadoop-2.2.0 libraries?

The simplest solution for Linux machines would ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 906 views
0 votes
1 answer

Hadoop: No Such Method Exception

Mapper and reducer classes need to be ...READ MORE

Oct 26, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 1,651 views
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

Aug 1, 2019 in Big Data Hadoop by Dinish
 • 17,412 views
0 votes
1 answer

How to fix corrupt HDFS FIles?

You can use hdfs fsck / to ...READ MORE

Oct 25, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,861 views
0 votes
3 answers

How to specify KeyValueTextInputFormat Separator in Hadoop-.20 api?

conf.set("key.value.separator.in.input.line", ","); Job job = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Rio
 • 1,585 views
0 votes
1 answer

Where is hadoop-test.jar in Hadoop2?

In Hadoop 1, testmapredsort is launched from AllTestDriver.java, which was ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 886 views
0 votes
1 answer

What is Hadoop Performance Tuning?

Is it consistent or one-off reading? Is ...READ MORE

Oct 24, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 769 views
0 votes
1 answer

Google Hadoop Filesystem Encryption

If you're asking if customer-supplied encryption keys ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 517 views
0 votes
1 answer

Is Java alone Sufficient for Hadoop?

In my day job, I've just spent ...READ MORE

Oct 18, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 451 views
0 votes
1 answer

What is -cp command in hadoop? How it works?

/user/cloudera/data1 is not a directory, it is ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 3,860 views
0 votes
1 answer

What is a data serialization system?

So when Hadoop was being written by ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,051 views
0 votes
1 answer

What is the best way to integrate SAS with Hadoop without losing the parallel processing capacity of Hadoop

One of the major pushes at SAS ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,972 views
0 votes
1 answer

How to configure Yarn from command line?

It's a bit non-intuitive, but it turns ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,664 views
0 votes
1 answer

Does Hadoop and Spark support iPv6 now?

It does not. As of today HADOOP-11890, which ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,760 views
0 votes
1 answer

Hadoop Cluster Node Setup.

Yes, at least in small clusters those ...READ MORE

Oct 15, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 437 views
0 votes
1 answer

What are the prerequisites to learn Hadoop in java perspective?

In my day job, I've just spent ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 608 views
0 votes
1 answer

POC for Hadoop in real time scenario

I'd like to point a few things. If ...READ MORE

Oct 11, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,463 views
0 votes
1 answer

PIG - Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Yes, it is a compatibility issue. in Hadoop ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 931 views
0 votes
1 answer

Setting textinputformat.record.delimiter in spark

I got this working with plain uncompressed ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 2,157 views
0 votes
1 answer

Is it possible to integrate Kdb+ and hadoop?

kdb is a column oriented database plus ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 783 views
0 votes
1 answer

Is MapR a substitute for MapReduce?

MapR is a commercial distribution of Apache ...READ MORE

Oct 10, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 965 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 508 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 16,558 views
0 votes
2 answers

Sqoop: Could not load mysql driver exception.

I add mysql driver for sqoop and it ...READ MORE

Mar 1, 2020 in Big Data Hadoop by Manh Quang
 • 180 points • 3,343 views
0 votes
1 answer

Hadoop on OSX “Unable to load realm info from SCDynamicStore”

Add the following to your hadoop-env.sh file: export ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 517 views
0 votes
1 answer

How can I calculate exact median with Apache Spark?

You need to sort RDD and take ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
 • 69,210 points • 4,109 views
0 votes
1 answer

How can I access S3/S3n from a local Hadoop 2.6 installation?

For some reason, the jar hadoop-aws-[version].jar which contains the ...READ MORE

Oct 3, 2018 in Big Data Hadoop by digger
 • 26,740 points • 5,111 views
0 votes
1 answer

What is Hadoop Speculative task execution?

One problem with the Hadoop system is ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 887 views
0 votes
1 answer

What is the difference between Writable & WritableComparable in Hadoop?

Writable in an interface in Hadoop and types ...READ MORE

Oct 3, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 7,313 views
0 votes
1 answer

out of Memory Error in Hadoop

You can assign more memory by editing ...READ MORE

Oct 1, 2018 in Big Data Hadoop by slayer
 • 29,350 points • 1,467 views
0 votes
1 answer

Iterate twice on values (MapReduce)

We have to cache the values from ...READ MORE

Oct 1, 2018 in Big Data Hadoop by digger
 • 26,740 points • 1,906 views
0 votes
2 answers

How to convert .txt file to Hadoop's sequence file format

import java.io.IOException; import java.net.URI; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IOUtils; import ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Sanjay
 • 3,202 views
0 votes
1 answer

Using Hadoop for Data Analytics.

Yes, your approach is correct - you ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 527 views
0 votes
1 answer

How to write a file in HDFS using Java Programming language?

Define the HADOOP_CONF_DIR environment variable to your Hadoop configuration ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,635 views
0 votes
1 answer

What should be the choice of database and what type of data format is suitable for Spark/hadoop?

Use Parquet. I'm not sure about CSV ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 781 views
0 votes
1 answer

When I try to connect to Hadoop, I get a error as -"Connecting to Resource Manager failed".

Make sure you've started Yarn. Use this ...READ MORE

Sep 28, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,159 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.