Most answered questions in Big Data Hadoop

0 votes
13 answers

What is the difference between Hadoop/HDFS & HBase?

HDFS is a distributed file system whereas ...READ MORE

Apr 26, 2019 in Big Data Hadoop by Arihar
• 160 points
32,972 views
+1 vote
12 answers

How to check size of HDFS directory?

It is the same syntax. Use the ...READ MORE

Jun 6, 2019 in Big Data Hadoop by Sowmya
46,938 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

Apr 29, 2019 in Big Data Hadoop by Gitika
• 65,910 points
29,053 views
0 votes
11 answers

What is the command to start Job history server in Hadoop 2.x & how to get its UI?

mr-jobhistory-daemon. sh start historyserver READ MORE

Jul 24, 2019 in Big Data Hadoop by Lokesh Singh
20,289 views
0 votes
11 answers

What is the default Namenode port of HDFS. Is it 8020 or 9000 or 50070?

8020/9000 hadoop namenode metadata port number. or ...READ MORE

Dec 31, 2018 in Big Data Hadoop by SM
61,077 views
+11 votes
11 answers

Hadoop “Unable to load native-hadoop library for your platform” warning

modify the glibc version.CentOS provides safe softwares ...READ MORE

Sep 10, 2018 in Big Data Hadoop by bug_seeker
• 15,520 points
70,690 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
106,779 views
0 votes
10 answers

Difference between single node & pseudo-distributed mode in Hadoop?

Single node is used for debugging the ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Mahisha
18,758 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

hadoop fs <args> fs is used for generic ...READ MORE

Mar 16, 2018 in Big Data Hadoop by anonymous
33,097 views
0 votes
10 answers

What is the difference between Mongodb and Hadoop?

MongoDB is a NoSQL database, whereas Hadoop is ...READ MORE

Jun 20, 2018 in Big Data Hadoop by jenny_code
11,658 views
0 votes
9 answers

Is there any way to check which Hadoop daemons are running?

use jps command, It will show all the running ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Rakesh
• 160 points
45,958 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

Dec 10, 2018 in Big Data Hadoop by Dasinto
26,022 views
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

Aug 1, 2019 in Big Data Hadoop by Dinish
17,746 views
+2 votes
5 answers

Not able to start hadoop dfs

You can re-install openssh-client and openssh-server: $ sudo ...READ MORE

Oct 25, 2018 in Big Data Hadoop by Jino
2,806 views
+2 votes
5 answers

How to transpose/pivot data in hive?

Below is also a way for Pivot SELECT ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Rahul
19,479 views
+3 votes
5 answers

Hadoop DistributedCache is deprecated - what is the preferred API?

I had the same problem. And not ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Rohan
2,092 views
0 votes
5 answers

What is the HDFS command to list all the files in HDFS according to the timestamp?

You can try filter using value in ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Lenny
71,934 views
+2 votes
4 answers

Datanode process not running in Hadoop

Run the following commands: Stop-all.sh start-dfs.sh start-yarn.sh mr-jobhistory-daemon.sh ...READ MORE

Oct 25, 2018 in Big Data Hadoop by Anand
17,813 views
+1 vote
3 answers

What is the best way to merge multi-part HDFS files into single file?

1. In order to merge two or ...READ MORE

Jul 29, 2019 in Big Data Hadoop by Tina
30,225 views
0 votes
3 answers

JDBC connection string for Netezza

HI.. Netezza is one of the widely used ...READ MORE

Oct 30, 2020 in Big Data Hadoop by SRI
3,141 views
0 votes
3 answers

Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver

I add mysql driver for sqoop and it ...READ MORE

Mar 1, 2020 in Big Data Hadoop by Manh Quang
• 180 points
7,034 views
0 votes
3 answers

How to change the delimiter in Sqoop?

--fields-terminated-by <char> READ MORE

Jun 25, 2019 in Big Data Hadoop by anonymous
10,235 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Mark
16,910 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

Using PySpark  hadoop = sc._jvm.org.apache.hadoop fs = hadoop.fs.FileSystem conf = ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Kiran
10,792 views
+1 vote
3 answers

HIVE: java.net.URISyntaxException

Change in hfs-site.xml this properties <name>hive.exec.scratchdir</name> <value>/tmp/hive-${user.name}</value> <name>hive.exec.local.scratchdir</name> <value>/tmp/${user.name}</value> <name>hive.downloaded.resources.dir</name> <value>/tmp/${user.name}_resources</value> <name>hive.scratch.dir.permission</name> ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Roshan
4,778 views
0 votes
3 answers

How to specify KeyValueTextInputFormat Separator in Hadoop-.20 api?

conf.set("key.value.separator.in.input.line", ","); Job job = new ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Rio
1,719 views
+4 votes
3 answers

Unable to find file error- while Streaming Hadoop

Is python installed running on the slaves that ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Vinith
3,937 views
+1 vote
3 answers

when i tried to run the hdfs namenode -format command...its failed to run..can anyone help me with this???

Hi Suriyaprakash, I guess the path for hadoop ...READ MORE

Jul 23, 2018 in Big Data Hadoop by avi_man_utd
11,773 views
0 votes
3 answers

How to connect Spark to a remote Hive server?

JDBC is not required here. Create a hive ...READ MORE

Mar 8, 2019 in Big Data Hadoop by Vijay Dixon
• 190 points
12,345 views
0 votes
3 answers

Cloudera Hadoop - Daemons not running

Please run below mentioned command. It will ...READ MORE

Aug 7, 2018 in Big Data Hadoop by Priyaj
• 58,090 points
2,470 views
0 votes
3 answers

Can we run Spark without using Hadoop?

No, you can run spark without hadoop. ...READ MORE

May 7, 2019 in Big Data Hadoop by pradeep
2,047 views
0 votes
3 answers

What are differences between NameNode and Secondary NameNode?

File metadata information is stored by Namenode ...READ MORE

Apr 8, 2019 in Big Data Hadoop by anonymous
14,733 views
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

Jul 1, 2019 in Big Data Hadoop by Ved Gupta
23,353 views
0 votes
2 answers

What is Hadoop used for?

Apache Hadoop is an open-source framework that ...READ MORE

Dec 10, 2021 in Big Data Hadoop by CoderGirl
• 500 points
586 views
+1 vote
2 answers

How to delete and update a record in Hive?

Hi, Earlier, there was no operation supported for ...READ MORE

Dec 18, 2020 in Big Data Hadoop by akhtar
• 38,240 points
6,316 views
0 votes
2 answers

How do I output the results of a HiveQL query to CSV?

Hi, The insert overwrite directory is running in ...READ MORE

Dec 18, 2020 in Big Data Hadoop by akhtar
• 38,240 points
8,048 views
0 votes
2 answers

How to know Hive and Hadoop versions from command prompt?

Hi, Hadoop and hive have their individual commands. ...READ MORE

Dec 18, 2020 in Big Data Hadoop by akhtar
• 38,240 points
9,452 views
0 votes
2 answers

Unable to start YARN daemons. Error: Could not find or load main class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

Hi@Akshitha, This seems like a path issue. you ...READ MORE

Nov 9, 2020 in Big Data Hadoop by MD
• 95,440 points
2,368 views
0 votes
2 answers
0 votes
2 answers

How to set properties for secondary namenode in Hadoop?

Secondary NameNode in HDFS Secondary NameNode in Hadoop ...READ MORE

Mar 31, 2020 in Big Data Hadoop by anonymous
3,869 views
+1 vote
2 answers

How to find previous records from a data set in Pig??

Hi, You can use ToDate() and SubtractDuration() function ...READ MORE

Jan 23, 2020 in Big Data Hadoop by MD
• 95,440 points
1,145 views
+1 vote
2 answers

Hadoop Learning Path

Hi! Since you have only 6 weeks, it's ...READ MORE

Nov 28, 2019 in Big Data Hadoop by Farheen
1,203 views
+1 vote
2 answers

Unable to create external table on HIVE

Hi @Ozer, This error occurred because there is ...READ MORE

Nov 7, 2019 in Big Data Hadoop by Richa
4,369 views
+1 vote
2 answers

How to find the database daily refresh time / sync up time

I use this code to get the ...READ MORE

Sep 17, 2019 in Big Data Hadoop by Raman
9,092 views
0 votes
2 answers

How to use scoop in a Java Program

There is a trick which worked out ...READ MORE

Sep 4, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
1,539 views
+1 vote
2 answers

Error: com/eng/mr/WordCountDriver : Unsupported major.minor version 52.0

The reason why you get this error ...READ MORE

Aug 6, 2019 in Big Data Hadoop by Venkita
• 240 points
610 views
+1 vote
2 answers

Hadoop: Error: Exception in thread “main” java.lang.UnsupportedClassVersionError: a

The reason for this error is because ...READ MORE

Aug 6, 2019 in Big Data Hadoop by Venkita
• 240 points
1,647 views
+1 vote
2 answers

Hadoop Administration

Make the following changes to your configuration ...READ MORE

Aug 5, 2019 in Big Data Hadoop by Simran
814 views
0 votes
2 answers

Load data into the hive table

Hi, We can use a normal insert query ...READ MORE

Oct 16, 2020 in Big Data Hadoop by MD
• 95,440 points
1,875 views
0 votes
2 answers

Not Able to read the file from hdfs location

Please make sure you connect to spark2-shell ...READ MORE

Jul 14, 2020 in Big Data Hadoop by Shantanu
• 190 points
1,975 views