Most viewed questions in Big Data Hadoop

0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Aditya
14,820 views
+10 votes
11 answers

Hadoop “Unable to load native-hadoop library for your platform” warning

modify the glibc version.CentOS provides safe softwares ...READ MORE

Sep 10, 2018 in Big Data Hadoop by bug_seeker
• 15,350 points
12,215 views
0 votes
11 answers

What is the default Namenode port of HDFS. Is it 8020 or 9000 or 50070?

8020/9000 hadoop namenode metadata port number. or ...READ MORE

Dec 31, 2018 in Big Data Hadoop by SM
11,592 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Kunal
10,444 views
0 votes
5 answers

What is the HDFS command to list all the files in HDFS according to the timestamp?

You can try filter using value in ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Lenny
9,661 views
0 votes
13 answers

What is the difference between Hadoop/HDFS & HBase?

HDFS is a distributed file system whereas ...READ MORE

Apr 26 in Big Data Hadoop by Arihar
• 160 points
9,584 views
0 votes
11 answers

How to check size of HDFS directory?

It is the same syntax. Use the ...READ MORE

Jun 6 in Big Data Hadoop by Sowmya
8,840 views
0 votes
9 answers

Is there any way to check which Hadoop daemons are running?

use jps command, It will show all the running ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Rakesh
• 160 points
8,058 views
0 votes
1 answer

How to groupBy/count then filter on count in Scala

I think the exception is caused because ...READ MORE

Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
7,662 views
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

Jul 1 in Big Data Hadoop by Ved Gupta
6,318 views
0 votes
1 answer

http://localhost:50070 not working in HADOOP

There are 2 case possible: 1. If you ...READ MORE

Apr 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
6,200 views
0 votes
1 answer

Writing File into HDFS using spark scala

The reason you are not able to ...READ MORE

Apr 5, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
5,725 views
0 votes
1 answer

Copy file from HDFS to the local file system

There are two possible ways to copy ...READ MORE

Mar 27, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
5,649 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

Dec 10, 2018 in Big Data Hadoop by Dasinto
5,425 views
+2 votes
5 answers

How to transpose/pivot data in hive?

Below is also a way for Pivot SELECT ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Rahul
5,257 views
0 votes
11 answers

What is the command to start Job history server in Hadoop 2.x & how to get its UI?

mr-jobhistory-daemon. sh start historyserver READ MORE

Jul 23 in Big Data Hadoop by Lokesh Singh
4,997 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

Apr 29 in Big Data Hadoop by Gitika
• 25,340 points
4,940 views
0 votes
1 answer

What is the default location of Hadoop log files & how can I change that location?

The default directory of Hadoop log file ...READ MORE

May 4, 2018 in Big Data Hadoop by Shubham
• 13,300 points
4,838 views
0 votes
2 answers

Function of “hadoop namenode -format” command

Namenode is the node in the Hadoop ...READ MORE

Feb 4 in Big Data Hadoop by Kanika
4,661 views
0 votes
1 answer

How to replace characters in string in hive?

You can do it using regexp_replace. This is ...READ MORE

Nov 29, 2018 in Big Data Hadoop by Omkar
• 67,620 points
4,149 views
0 votes
10 answers

Difference between single node & pseudo-distributed mode in Hadoop?

Both are the same thing but single ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Vilola
3,833 views
0 votes
1 answer

How to retrieve the list of sql (Hive QL) commands that has been executed in a hadoop cluster?

Well, what you can do is use ...READ MORE

Apr 4, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
3,493 views
0 votes
2 answers

hadoop copy a local file system folder to HDFS

There's a typo in your command: "hadopp". ...READ MORE

Feb 4 in Big Data Hadoop by Lohith
3,419 views
0 votes
1 answer

Installing Zookeeper on Windows

Follow the below-mentioned steps to install Zookeeper ...READ MORE

Feb 22 in Big Data Hadoop by Omkar
• 67,620 points
3,328 views
0 votes
1 answer

How to count lines in a file on hdfs command?

Use the below commands: Total number of files: hadoop ...READ MORE

Aug 10, 2018 in Big Data Hadoop by Neha
• 6,280 points
3,187 views
0 votes
1 answer

Spark - load CSV file as DataFrame?

spark-csv is part of core Spark functionality ...READ MORE

Sep 25, 2018 in Big Data Hadoop by slayer
• 29,170 points
3,103 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
• 67,620 points
2,971 views
0 votes
1 answer

Bucketing vs Partitioning in HIve

Partition divides large amount of data into ...READ MORE

Jul 9, 2018 in Big Data Hadoop by Data_Nerd
• 2,360 points
2,892 views
0 votes
1 answer

hadoop: command not found

The reason you are getting hadoop as ...READ MORE

Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
2,850 views
0 votes
1 answer

Difference between Application Master and Application Manager?

The Application Master is responsible for the ...READ MORE

Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
2,849 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
• 6,280 points
2,690 views
0 votes
2 answers

Difference between HDFS and NFS

In NFS, the data is stored only ...READ MORE

Feb 4 in Big Data Hadoop by Kumar
2,651 views
0 votes
10 answers

What is the difference between Mongodb and Hadoop?

Apart from the similarity that they are ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Deeraj
2,529 views
0 votes
1 answer

How can I list NameNode & DataNodes from any machine in the Hadoop cluster?

You can use the jps command to ...READ MORE

Apr 18, 2018 in Big Data Hadoop by coldcode
• 2,020 points
2,506 views
0 votes
1 answer

How to uninstall Yarn?

This is maybe because you didn't use ...READ MORE

Feb 4 in Big Data Hadoop by Omkar
• 67,620 points
2,442 views
0 votes
1 answer

What are SUCCESS and part-r-00000 files in Hadoop?

Yes, both the files i.e. SUCCESS and ...READ MORE

Apr 12, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
2,436 views
0 votes
1 answer

“no such file or directory" in case of hadoop fs -ls

The behaviour that you are seeing is ...READ MORE

May 9, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points

edited May 9, 2018 by nitinrawat895 2,415 views
0 votes
1 answer

How to install Hive on Windows Operating System?

First, download Hive from the following link: http://www.apache.org/dyn/closer.cgi/hive/ Next, ...READ MORE

Feb 26 in Big Data Hadoop by Kirti
2,360 views
0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

Jan 28 in Big Data Hadoop by Omkar
• 67,620 points
2,317 views
0 votes
1 answer

Hadoop Hive: How to split string in Hive?

You can use the split function along ...READ MORE

Nov 6, 2018 in Big Data Hadoop by Omkar
• 67,620 points
2,280 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 67,620 points
2,271 views
0 votes
2 answers

How to get column names from Hive table?

To get all the columns of a ...READ MORE

Jun 4 in Big Data Hadoop by anonymous

edited Jun 4 by Omkar 2,254 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

import org.apache.hadoop.fs.{FileSystem,Path} FileSystem.get( sc.hadoopConfiguration ).listStatus( new Path("hdfs:///tmp")).foreach( ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Komal
2,237 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

This seems like a path issue. Add the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
• 67,620 points
2,183 views
0 votes
1 answer

After installing Hadoop on Windows7, NameNode failed to start

You need to install Hadoop winutils to ...READ MORE

May 3, 2018 in Big Data Hadoop by Shubham
• 13,300 points
2,157 views
0 votes
1 answer

How to execute python script in hadoop file system (hdfs)?

If you are simply looking to distribute ...READ MORE

Sep 19, 2018 in Big Data Hadoop by digger
• 26,550 points
2,092 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Mark
2,091 views
0 votes
1 answer

How to add user in supergroup of hdfs in linux?

Yes , now i have whole idea ...READ MORE

Sep 20, 2018 in Big Data Hadoop by Frankie
• 9,810 points
2,082 views
–1 vote
1 answer

What is the use of fsimage in hadoop?

The NameNode stores modifications to the file ...READ MORE

Dec 20, 2018 in Big Data Hadoop by Omkar
• 67,620 points
2,039 views
0 votes
1 answer

How to move data from Oracle database to Hadoop?

Yes, you heard it correctly. Apache Sqoop is ...READ MORE

Apr 11, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
2,011 views