Most viewed questions in Big Data Hadoop

+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Aditya
18,463 views
0 votes
11 answers

What is the default Namenode port of HDFS. Is it 8020 or 9000 or 50070?

8020/9000 hadoop namenode metadata port number. or ...READ MORE

Dec 31, 2018 in Big Data Hadoop by SM
14,742 views
+10 votes
11 answers

Hadoop “Unable to load native-hadoop library for your platform” warning

modify the glibc version.CentOS provides safe softwares ...READ MORE

Sep 10, 2018 in Big Data Hadoop by bug_seeker
• 15,360 points
14,624 views
0 votes
13 answers

What is the difference between Hadoop/HDFS & HBase?

HDFS is a distributed file system whereas ...READ MORE

Apr 26 in Big Data Hadoop by Arihar
• 160 points
11,956 views
0 votes
5 answers

What is the HDFS command to list all the files in HDFS according to the timestamp?

You can try filter using value in ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Lenny
11,945 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Kunal
11,707 views
0 votes
11 answers

How to check size of HDFS directory?

It is the same syntax. Use the ...READ MORE

Jun 6 in Big Data Hadoop by Sowmya
10,690 views
0 votes
9 answers

Is there any way to check which Hadoop daemons are running?

use jps command, It will show all the running ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Rakesh
• 160 points
9,981 views
0 votes
1 answer

How to groupBy/count then filter on count in Scala

I think the exception is caused because ...READ MORE

Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
9,798 views
+1 vote
1 answer

http://localhost:50070 not working in HADOOP

There are 2 case possible: 1. If you ...READ MORE

Apr 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
7,709 views
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

Jul 1 in Big Data Hadoop by Ved Gupta
7,689 views
0 votes
1 answer

Copy file from HDFS to the local file system

There are two possible ways to copy ...READ MORE

Mar 27, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
6,949 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

Dec 10, 2018 in Big Data Hadoop by Dasinto
6,624 views
0 votes
1 answer

Writing File into HDFS using spark scala

The reason you are not able to ...READ MORE

Apr 5, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
6,604 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

Apr 29 in Big Data Hadoop by Gitika
• 25,420 points
6,429 views
+2 votes
5 answers

How to transpose/pivot data in hive?

Below is also a way for Pivot SELECT ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Rahul
6,391 views
0 votes
1 answer

What is the default location of Hadoop log files & how can I change that location?

The default directory of Hadoop log file ...READ MORE

May 4, 2018 in Big Data Hadoop by Shubham
• 13,350 points
6,147 views
0 votes
11 answers

What is the command to start Job history server in Hadoop 2.x & how to get its UI?

mr-jobhistory-daemon. sh start historyserver READ MORE

Jul 23 in Big Data Hadoop by Lokesh Singh
5,763 views
0 votes
2 answers

Function of “hadoop namenode -format” command

Namenode is the node in the Hadoop ...READ MORE

Feb 4 in Big Data Hadoop by Kanika
5,605 views
0 votes
1 answer

How to replace characters in string in hive?

You can do it using regexp_replace. This is ...READ MORE

Nov 29, 2018 in Big Data Hadoop by Omkar
• 68,480 points
5,569 views
0 votes
1 answer

Installing Zookeeper on Windows

Follow the below-mentioned steps to install Zookeeper ...READ MORE

Feb 22 in Big Data Hadoop by Omkar
• 68,480 points
5,509 views
0 votes
10 answers

Difference between single node & pseudo-distributed mode in Hadoop?

Both are the same thing but single ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Vilola
4,915 views
0 votes
2 answers

hadoop copy a local file system folder to HDFS

There's a typo in your command: "hadopp". ...READ MORE

Feb 4 in Big Data Hadoop by Lohith
4,302 views
0 votes
1 answer

How to retrieve the list of sql (Hive QL) commands that has been executed in a hadoop cluster?

Well, what you can do is use ...READ MORE

Apr 4, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
4,300 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
• 6,280 points
4,098 views
0 votes
1 answer

How to count lines in a file on hdfs command?

Use the below commands: Total number of files: hadoop ...READ MORE

Aug 10, 2018 in Big Data Hadoop by Neha
• 6,280 points
3,978 views
0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

Jan 28 in Big Data Hadoop by Omkar
• 68,480 points
3,909 views
0 votes
1 answer

Bucketing vs Partitioning in HIve

Partition divides large amount of data into ...READ MORE

Jul 9, 2018 in Big Data Hadoop by Data_Nerd
• 2,360 points
3,895 views
0 votes
1 answer

How to uninstall Yarn?

This is maybe because you didn't use ...READ MORE

Feb 4 in Big Data Hadoop by Omkar
• 68,480 points
3,805 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
• 68,480 points
3,648 views
0 votes
1 answer

Spark - load CSV file as DataFrame?

spark-csv is part of core Spark functionality ...READ MORE

Sep 25, 2018 in Big Data Hadoop by slayer
• 29,240 points
3,555 views
0 votes
1 answer

How to install Hive on Windows Operating System?

First, download Hive from the following link: http://www.apache.org/dyn/closer.cgi/hive/ Next, ...READ MORE

Feb 26 in Big Data Hadoop by Kirti
3,483 views
0 votes
1 answer

Difference between Application Master and Application Manager?

The Application Master is responsible for the ...READ MORE

Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
3,445 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

import org.apache.hadoop.fs.{FileSystem,Path} FileSystem.get( sc.hadoopConfiguration ).listStatus( new Path("hdfs:///tmp")).foreach( ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Komal
3,406 views
0 votes
2 answers

Difference between HDFS and NFS

In NFS, the data is stored only ...READ MORE

Feb 4 in Big Data Hadoop by Kumar
3,334 views
0 votes
1 answer

hadoop: command not found

The reason you are getting hadoop as ...READ MORE

Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
3,250 views
–1 vote
1 answer

What is the use of fsimage in hadoop?

The NameNode stores modifications to the file ...READ MORE

Dec 20, 2018 in Big Data Hadoop by Omkar
• 68,480 points
3,216 views
0 votes
1 answer

Hadoop Hive: How to split string in Hive?

You can use the split function along ...READ MORE

Nov 6, 2018 in Big Data Hadoop by Omkar
• 68,480 points
3,159 views
0 votes
1 answer

How can I list NameNode & DataNodes from any machine in the Hadoop cluster?

You can use the jps command to ...READ MORE

Apr 18, 2018 in Big Data Hadoop by coldcode
• 2,040 points
3,139 views
0 votes
2 answers

How to get column names from Hive table?

To get all the columns of a ...READ MORE

Jun 4 in Big Data Hadoop by anonymous

edited Jun 4 by Omkar 3,032 views
0 votes
1 answer

What are SUCCESS and part-r-00000 files in Hadoop?

Yes, both the files i.e. SUCCESS and ...READ MORE

Apr 12, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
2,991 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 68,480 points
2,940 views
0 votes
10 answers

What is the difference between Mongodb and Hadoop?

Apart from the similarity that they are ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Deeraj
2,867 views
0 votes
1 answer

Managed Table vs External Table

External table is created for external use ...READ MORE

Dec 13, 2018 in Big Data Hadoop by Omkar
• 68,480 points
2,826 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

This seems like a path issue. Add the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
• 68,480 points
2,780 views
0 votes
1 answer

“no such file or directory" in case of hadoop fs -ls

The behaviour that you are seeing is ...READ MORE

May 9, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points

edited May 9, 2018 by nitinrawat895 2,774 views
0 votes
1 answer

How to execute python script in hadoop file system (hdfs)?

If you are simply looking to distribute ...READ MORE

Sep 19, 2018 in Big Data Hadoop by digger
• 26,600 points
2,746 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Mark
2,727 views
0 votes
1 answer

Spark : java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream

Create a spark-env.cmd file looking like this @echo off set ...READ MORE

Jan 2 in Big Data Hadoop by Omkar
• 68,480 points
2,578 views
+2 votes
2 answers

Create HIVE Table with multi character delimiter

FILELDS TERMINATED BY does not support multi-character delimiters. ...READ MORE

Oct 3, 2018 in Big Data Hadoop by slayer
• 29,240 points
2,562 views