Most viewed questions in Big Data Hadoop

+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Aditya
29,637 views
+10 votes
11 answers

Hadoop “Unable to load native-hadoop library for your platform” warning

modify the glibc version.CentOS provides safe softwares ...READ MORE

Sep 10, 2018 in Big Data Hadoop by bug_seeker
• 15,390 points
23,737 views
0 votes
11 answers

What is the default Namenode port of HDFS. Is it 8020 or 9000 or 50070?

8020/9000 hadoop namenode metadata port number. or ...READ MORE

Dec 31, 2018 in Big Data Hadoop by SM
22,678 views
0 votes
5 answers

What is the HDFS command to list all the files in HDFS according to the timestamp?

You can try filter using value in ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Lenny
19,605 views
0 votes
11 answers

How to check size of HDFS directory?

It is the same syntax. Use the ...READ MORE

Jun 6, 2019 in Big Data Hadoop by Sowmya
17,093 views
0 votes
13 answers

What is the difference between Hadoop/HDFS & HBase?

HDFS is a distributed file system whereas ...READ MORE

Apr 26, 2019 in Big Data Hadoop by Arihar
• 160 points
16,825 views
0 votes
1 answer

How to groupBy/count then filter on count in Scala

I think the exception is caused because ...READ MORE

Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
15,562 views
0 votes
9 answers

Is there any way to check which Hadoop daemons are running?

use jps command, It will show all the running ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Rakesh
• 160 points
15,348 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Kunal
14,671 views
0 votes
1 answer

Installing Zookeeper on Windows

Follow the below-mentioned steps to install Zookeeper ...READ MORE

Feb 22, 2019 in Big Data Hadoop by Omkar
• 69,040 points
12,237 views
+1 vote
1 answer

http://localhost:50070 not working in HADOOP

There are 2 case possible: 1. If you ...READ MORE

Apr 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,920 points
11,654 views
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

Jul 1, 2019 in Big Data Hadoop by Ved Gupta
11,404 views
+2 votes
5 answers

How to transpose/pivot data in hive?

Below is also a way for Pivot SELECT ...READ MORE

Oct 12, 2018 in Big Data Hadoop by Rahul
9,536 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

Apr 29, 2019 in Big Data Hadoop by Gitika
• 28,810 points
9,447 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

Dec 10, 2018 in Big Data Hadoop by Dasinto
9,355 views
0 votes
1 answer

Copy file from HDFS to the local file system

There are two possible ways to copy ...READ MORE

Mar 27, 2018 in Big Data Hadoop by nitinrawat895
• 10,920 points
9,342 views
0 votes
1 answer

Writing File into HDFS using spark scala

The reason you are not able to ...READ MORE

Apr 5, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
9,227 views
0 votes
1 answer

What is the default location of Hadoop log files & how can I change that location?

The default directory of Hadoop log file ...READ MORE

May 4, 2018 in Big Data Hadoop by Shubham
• 13,380 points
8,901 views
0 votes
1 answer

How to replace characters in string in hive?

You can do it using regexp_replace. This is ...READ MORE

Nov 29, 2018 in Big Data Hadoop by Omkar
• 69,040 points
8,861 views
0 votes
1 answer

How to uninstall Yarn?

This is maybe because you didn't use ...READ MORE

Feb 4, 2019 in Big Data Hadoop by Omkar
• 69,040 points
8,646 views
0 votes
2 answers

Function of “hadoop namenode -format” command

Namenode is the node in the Hadoop ...READ MORE

Feb 4, 2019 in Big Data Hadoop by Kanika
8,506 views
0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

Jan 28, 2019 in Big Data Hadoop by Omkar
• 69,040 points
8,366 views
0 votes
2 answers

hadoop copy a local file system folder to HDFS

There's a typo in your command: "hadopp". ...READ MORE

Feb 4, 2019 in Big Data Hadoop by Lohith
7,980 views
0 votes
11 answers

What is the command to start Job history server in Hadoop 2.x & how to get its UI?

mr-jobhistory-daemon. sh start historyserver READ MORE

Jul 23, 2019 in Big Data Hadoop by Lokesh Singh
7,899 views
0 votes
1 answer

How to convert a string to timestamp with milliseconds in Hive?

 concatenation of substrings using the following code: select ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Neha
• 6,280 points
7,605 views
0 votes
1 answer

ERROR 1142 (42000): SELECT command denied to user ''@'localhost' for table 'user'

Hey, This is a mysql command line error ...READ MORE

Jun 19, 2019 in Big Data Hadoop by Gitika
• 28,810 points
7,184 views
0 votes
1 answer

Bucketing vs Partitioning in HIve

Partition divides large amount of data into ...READ MORE

Jul 9, 2018 in Big Data Hadoop by Data_Nerd
• 2,370 points
7,078 views
0 votes
10 answers

Difference between single node & pseudo-distributed mode in Hadoop?

Single node is used for debugging the ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Mahisha
6,818 views
0 votes
1 answer

How to install Hive on Windows Operating System?

First, download Hive from the following link: http://www.apache.org/dyn/closer.cgi/hive/ Next, ...READ MORE

Feb 26, 2019 in Big Data Hadoop by Kirti
6,611 views
0 votes
1 answer

How to count lines in a file on hdfs command?

Use the below commands: Total number of files: hadoop ...READ MORE

Aug 10, 2018 in Big Data Hadoop by Neha
• 6,280 points
6,530 views
0 votes
1 answer

How to retrieve the list of sql (Hive QL) commands that has been executed in a hadoop cluster?

Well, what you can do is use ...READ MORE

Apr 4, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
6,425 views
–1 vote
1 answer

What is the use of fsimage in hadoop?

The NameNode stores modifications to the file ...READ MORE

Dec 20, 2018 in Big Data Hadoop by Omkar
• 69,040 points
6,319 views
0 votes
2 answers

How to get column names from Hive table?

To get all the columns of a ...READ MORE

Jun 4, 2019 in Big Data Hadoop by anonymous

edited Jun 4, 2019 by Omkar 6,155 views
+1 vote
1 answer

What is the best way to merge multi-part HDFS files into single file?

1. In order to merge two or ...READ MORE

Jul 29, 2019 in Big Data Hadoop by Tina
5,992 views
+1 vote
1 answer

How to limit the number of rows per each item in a Hive QL?

SELECT a_id, b, c, count(*) as sumrequests FROM ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Omkar
• 69,040 points
5,911 views
0 votes
1 answer

How to Access Hive via Python?

The easiest way is to use PyHive. To ...READ MORE

Oct 9, 2018 in Big Data Hadoop by Omkar
• 69,040 points
5,900 views
0 votes
1 answer

Hadoop Hive: How to split string in Hive?

You can use the split function along ...READ MORE

Nov 6, 2018 in Big Data Hadoop by Omkar
• 69,040 points
5,433 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

import org.apache.hadoop.fs.{FileSystem,Path} FileSystem.get( sc.hadoopConfiguration ).listStatus( new Path("hdfs:///tmp")).foreach( ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Komal
5,316 views
0 votes
1 answer

Replace Column names and datatype in Hive

You can replace the column name and ...READ MORE

May 21, 2019 in Big Data Hadoop by John
5,157 views
0 votes
1 answer

Difference between Application Master and Application Manager?

The Application Master is responsible for the ...READ MORE

Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 10,920 points
5,131 views
0 votes
1 answer

Managed Table vs External Table

External table is created for external use ...READ MORE

Dec 13, 2018 in Big Data Hadoop by Omkar
• 69,040 points
5,107 views
0 votes
2 answers

How can I list NameNode & DataNodes from any machine in the Hadoop cluster?

You can browse hadoop page from any ...READ MORE

Jan 23 in Big Data Hadoop by MD
• 21,340 points
4,997 views
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

Aug 1, 2019 in Big Data Hadoop by Dinish
4,949 views
0 votes
3 answers

Spark Scala: How to list all folders in directory

val spark = SparkSession.builder().appName("Demo").getOrCreate() val path = new ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Mark
4,892 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,040 points
4,881 views
0 votes
2 answers

Difference between HDFS and NFS

In NFS, the data is stored only ...READ MORE

Feb 4, 2019 in Big Data Hadoop by Kumar
4,871 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

This seems like a path issue. Add the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
• 69,040 points
4,613 views
0 votes
1 answer

Hadoop hdfs "permission denied" error when I try to make new directory

This is because you dont have enough ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
• 69,040 points
4,600 views
0 votes
10 answers

What is the difference between Mongodb and Hadoop?

Apart from the similarity that they are ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Deeraj
4,598 views
0 votes
1 answer

How to execute python script in hadoop file system (hdfs)?

If you are simply looking to distribute ...READ MORE

Sep 19, 2018 in Big Data Hadoop by digger
• 26,660 points
4,581 views