Most viewed questions in Big Data Hadoop

+1 vote
1 answer

How to append data to a parquet file?

Try using Spark API to append the ...READ MORE

Jan 11, 2019 in Big Data Hadoop by Omkar
• 69,230 points
16,098 views
0 votes
1 answer

Where does HDFS stores data on the local file system?

First find the Hadoop directory present in ...READ MORE

May 8, 2018 in Big Data Hadoop by Shubham
• 13,490 points
16,087 views
0 votes
1 answer

Hive complex data types

There are three complex types in hive, arrays: ...READ MORE

Jul 5, 2019 in Big Data Hadoop by Gani
15,501 views
0 votes
1 answer

Hadoop hdfs "permission denied" error when I try to make new directory

This is because you dont have enough ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
• 69,230 points
15,214 views
0 votes
1 answer

Why am I getting error as hdfs dfs -mkdir, No such file or directory when I create a directory?

It is because the parent directories do ...READ MORE

Sep 10, 2018 in Big Data Hadoop by Frankie
• 9,830 points
14,986 views
0 votes
1 answer

How to install Hive on Windows Operating System?

First, download Hive from the following link: http://www.apache.org/dyn/closer.cgi/hive/ Next, ...READ MORE

Feb 27, 2019 in Big Data Hadoop by Kirti
14,808 views
0 votes
3 answers

What are differences between NameNode and Secondary NameNode?

File metadata information is stored by Namenode ...READ MORE

Apr 8, 2019 in Big Data Hadoop by anonymous
14,704 views
0 votes
2 answers

Hadoop is not able to find jps command

jps is actually not a command of ...READ MORE

Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,490 points
14,662 views
0 votes
2 answers

Difference between HDFS and NFS

In NFS, the data is stored only ...READ MORE

Feb 4, 2019 in Big Data Hadoop by Kumar
14,522 views
0 votes
2 answers

Why we use --split by command in Sqoop?

In simple explanation, When specify SPLIT_BY only ...READ MORE

Feb 6, 2020 in Big Data Hadoop by Ramji Sridaran
14,507 views
0 votes
1 answer

What is the main purpose of HDFS fsck command?

HDFS fsck is used to check the ...READ MORE

Apr 10, 2019 in Big Data Hadoop by Gitika
• 65,910 points

edited Apr 12, 2019 by Gitika 14,432 views
0 votes
1 answer

Import JSON file into hive

There are two ways to load json ...READ MORE

Jul 15, 2019 in Big Data Hadoop by Guru
14,319 views
+2 votes
2 answers

Create HIVE Table with multi character delimiter

FILELDS TERMINATED BY does not support multi-character delimiters. ...READ MORE

Oct 3, 2018 in Big Data Hadoop by slayer
• 29,360 points
14,205 views
0 votes
1 answer

Difference between Application Master and Application Manager?

The Application Master is responsible for the ...READ MORE

Apr 10, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
14,145 views
0 votes
1 answer

How to retrieve the list of sql (Hive QL) commands that has been executed in a hadoop cluster?

Well, what you can do is use ...READ MORE

Apr 4, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
13,906 views
0 votes
1 answer

hadoop: command not found

The reason you are getting hadoop as ...READ MORE

Mar 28, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
13,848 views
0 votes
2 answers

Hadoop error: Input path does not exist

I was able to fix the issue. ...READ MORE

Jan 13, 2019 in Big Data Hadoop by Ponezhil Ramu
13,818 views
0 votes
1 answer

Hadoop HDFS: How to delete old files from HDFS?

You can use commands like this: hdfs dfs ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
• 69,230 points
13,777 views
0 votes
1 answer

How to run mapreduce program in terminal?

You can reference the below steps: Step 1: ...READ MORE

Jan 31, 2019 in Big Data Hadoop by Srishti
13,434 views
0 votes
1 answer

Difference between Cluster and nodes?

In Hadoop distributed system, Node is a single ...READ MORE

Apr 9, 2019 in Big Data Hadoop by Gitika
• 65,910 points
13,389 views
0 votes
1 answer

Impala concatenate string.

Yes, you can concatenate two strings using ...READ MORE

Jun 6, 2019 in Big Data Hadoop by Raghu
13,385 views
0 votes
1 answer

How to execute python script in hadoop file system (hdfs)?

If you are simply looking to distribute ...READ MORE

Sep 19, 2018 in Big Data Hadoop by digger
• 26,740 points
13,222 views
0 votes
1 answer
0 votes
1 answer

ERROR 1044 (42000): Access denied for user ''@'localhost' to database 'oozie'

Hey, I got the same error and it ...READ MORE

Jun 19, 2019 in Big Data Hadoop by Gitika
• 65,910 points
13,124 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,230 points
13,052 views
0 votes
1 answer

Failed to start namenode in hadoop

Change your following properties in hdfs-site.xml <property> ...READ MORE

Jan 12, 2019 in Big Data Hadoop by Omkar
• 69,230 points
12,884 views
0 votes
1 answer

Different Hadoop Modes

Hadoop can run in 3 different modes. 1. ...READ MORE

Jun 13, 2018 in Big Data Hadoop by shams
• 3,670 points
12,843 views
0 votes
1 answer

How to create an empty file in Hadoop?

Hi@akhtar, You can create an empty file in ...READ MORE

Oct 13, 2020 in Big Data Hadoop by MD
• 95,440 points
12,804 views
0 votes
1 answer

Managed Table vs External Table

External table is created for external use ...READ MORE

Dec 14, 2018 in Big Data Hadoop by Omkar
• 69,230 points
12,733 views
0 votes
3 answers

How to connect Spark to a remote Hive server?

JDBC is not required here. Create a hive ...READ MORE

Mar 8, 2019 in Big Data Hadoop by Vijay Dixon
• 190 points
12,343 views
0 votes
1 answer

Difference between put and copyFromLocal

-Put and -copyFromLocal is almost same command ...READ MORE

Jul 29, 2019 in Big Data Hadoop by Neeraj
12,286 views
0 votes
1 answer

What are SUCCESS and part-r-00000 files in Hadoop?

Yes, both the files i.e. SUCCESS and ...READ MORE

Apr 12, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
12,161 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,230 points
12,123 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

This seems like a path issue. Add the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
• 69,230 points
12,123 views
+1 vote
3 answers

when i tried to run the hdfs namenode -format command...its failed to run..can anyone help me with this???

Hi Suriyaprakash, I guess the path for hadoop ...READ MORE

Jul 23, 2018 in Big Data Hadoop by avi_man_utd
11,755 views
0 votes
1 answer

Spark : java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream

Create a spark-env.cmd file looking like this @echo off set ...READ MORE

Jan 2, 2019 in Big Data Hadoop by Omkar
• 69,230 points
11,653 views
0 votes
1 answer

Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)

This error usually occurs when the HiveServer2 ...READ MORE

Jun 4, 2019 in Big Data Hadoop by Ram
11,650 views
0 votes
10 answers

What is the difference between Mongodb and Hadoop?

MongoDB is a NoSQL database, whereas Hadoop is ...READ MORE

Jun 20, 2018 in Big Data Hadoop by jenny_code
11,643 views
0 votes
1 answer

Hadoop Hive: How to split string in Hive?

You can use the split function along ...READ MORE

Nov 6, 2018 in Big Data Hadoop by Omkar
• 69,230 points
11,557 views
+1 vote
1 answer

How can we list files in HDFS directory as per timestamp?

No, there is no other option to ...READ MORE

May 8, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
11,546 views
0 votes
1 answer

How to rename the Hive table?

Hi@akhtar, You can rename the table name in ...READ MORE

Oct 19, 2020 in Big Data Hadoop by MD
• 95,440 points
11,458 views
+1 vote
1 answer

Why minimum 3 Journal Nodes are required in Hadoop HA architecture?

Initially in Hadoop 1.x, the NameNode was ...READ MORE

Apr 20, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
11,373 views
0 votes
2 answers

How can I list NameNode & DataNodes from any machine in the Hadoop cluster?

You can browse hadoop page from any ...READ MORE

Jan 23, 2020 in Big Data Hadoop by MD
• 95,440 points
11,320 views
0 votes
1 answer

How to list files in hdfs that contains a specific string?

Yes, you can do this. You can ...READ MORE

Jan 27, 2019 in Big Data Hadoop by Omkar
• 69,230 points
11,207 views
0 votes
1 answer

Copy file from local to hdfs from the spark job in yarn mode

Please refer to the below code: import org.apache.hadoop.conf.Configuration import ...READ MORE

Jul 16, 2019 in Big Data Hadoop by Raj
11,075 views
0 votes
1 answer

How to extract only few lines of data from HDFS?

Here also in case of Hadoop, it is ...READ MORE

May 2, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
11,040 views
0 votes
1 answer

ERROR: Cannot set priority of datanode process

Hi@akhtar, You need to set JAVA_HOME and HADOOP_HOME ...READ MORE

Oct 15, 2020 in Big Data Hadoop by MD
• 95,440 points
11,012 views
0 votes
1 answer

How to add user in supergroup of hdfs in linux?

Yes , now i have whole idea ...READ MORE

Sep 21, 2018 in Big Data Hadoop by Frankie
• 9,830 points
10,823 views
0 votes
3 answers

Hadoop Spark: How to iterate hdfs directories?

Using PySpark  hadoop = sc._jvm.org.apache.hadoop fs = hadoop.fs.FileSystem conf = ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Kiran
10,780 views