Most answered questions in Big Data Hadoop

+1 vote
1 answer

How to get status of hdfs directory using python?

import commands hdir_list = commands.getoutput('hadoop fs -ls hdfs: ...READ MORE

Dec 7, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,443 views
+1 vote
1 answer

How to write file in hdfs using python?

#!/usr/bin/python from subprocess import Popen, PIPE cat = Popen(["hadoop", ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 8,111 views
0 votes
1 answer

How to create new directory in hdfs using java code?

Try this: { Configuration config ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,513 views
0 votes
1 answer

Not able to install hadoop using maven

First make sure you have ant installed ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,273 views
0 votes
1 answer

How to use jar file without putting it in hdfs?

If by "using", you mean distributing it, ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 572 views
0 votes
1 answer

How to concatenate hdfs files and store in output file?

You can use a combination of cat and put command. Something ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,589 views
0 votes
1 answer

Start-dfs.sh daemons not starting. localhost: mkdir: cannot create directory `/user': Permission denied

Make the following changes to the hadoop-env.sh ...READ MORE

Dec 6, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,800 views
0 votes
1 answer

Hadoop sqoop import not working. Error: ERROR manager.SqlManager: Error reading from database

In the command, try mentioning the driver ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,901 views
0 votes
1 answer

Not able to create Hive table from HDFS file

You dont have to specify the file name ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,311 views
0 votes
1 answer

Hadoop: Can not start namenode: Unable to load native-hadoop library for your platform

The entries in your .bashrc file looks ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,357 views
0 votes
1 answer

How to move Word and PDF documents to Hadoop HDFS?

Try with below commands: hadoop fs -copyFromLocal <localsrc> ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 841 views
0 votes
1 answer

Differentiate: hadoop fs -text vs hadoop fs -cat vs hadoop fs -get

The main difference between -cat and -text ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 4,045 views
0 votes
1 answer

Hadoop mapreduce error: apache.hadoop.mapred.FileAlreadyExistsException: Output directory

Seems like the output file already exists. ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,257 views
0 votes
1 answer

Hadoop Hive: Correlating expression cannot contain unqualified column references

Try the following command and see if ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,217 views
0 votes
1 answer

Cant find slave on ResourceManager Web UI

Seems like it is running on default ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,343 views
0 votes
1 answer

Why Hadoop is not implemented using MPI?

 One of the big features of Hadoop/map-reduce ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 618 views
0 votes
1 answer

How to get absolute path of files in a directory in Hadoop?

You can use HDFS API like the ...READ MORE

Dec 4, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 4,247 views
0 votes
1 answer

Unable to set JAVA_HOME path.

Check if bin/start-all.sh doesn't override JAVA_HOME put echo ...READ MORE

Dec 1, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,293 views
+1 vote
1 answer

How to limit the number of rows per each item in a Hive QL?

SELECT a_id, b, c, count(*) as sumrequests FROM ...READ MORE

Dec 1, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 26,144 views
0 votes
1 answer

Hive:Unable to insert values to an array column using "insert".

make a dummy table which has at least one row. INSERT ...READ MORE

Dec 1, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 7,949 views
0 votes
1 answer

Hadoop Mapper is failed because of the following error - “Container killed by the ApplicationMaster”

I suggest you include below property files ...READ MORE

Dec 1, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,799 views
0 votes
1 answer

Unable to move Hadoop Hive tables from Source to Destination.

First, check the permissions of HDFS Directory ...READ MORE

Dec 1, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,947 views
0 votes
1 answer

How to Upgrade Apache Hadoop from 2.4.1 to 2.6.0

If the downtime is not an issue, ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,279 views
0 votes
1 answer

Is Data locality concept applicable for Reducers also?

No, Data-Locality concept applies to MAPPERS only. Reducers ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,213 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class com.sun.tools.javac.Main

It looks like the path is not ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,020 views
0 votes
1 answer

How to replace characters in string in hive?

You can do it using regexp_replace. This is ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 16,853 views
0 votes
1 answer

Hadoop Job Tracker problem. I cannot connect to http://localhost:50030/

I think you are using new version ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,591 views
0 votes
1 answer

How to find the number of blocks a hdfs file is divided into?

Yes. you can use the hadoop fsck command to do ...READ MORE

Nov 30, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,521 views
0 votes
1 answer

How to delete a file from Hadoop hdfs using java?

Try this: FileSystem fs = FileSystem.get(getConf()); fs.delete(new ...READ MORE

Nov 28, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,391 views
0 votes
1 answer

Hive: How to know the path of a created table?

You can use the DESCRIBE command to ...READ MORE

Nov 28, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,842 views
0 votes
1 answer

Free Hadoop Cluster Experiments

I doubt if there is something which ...READ MORE

Nov 27, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,396 views
0 votes
1 answer

How to include third party library in Python MapReduce?

Problem has been solved by zipimport. Then I zip chardet to ...READ MORE

Nov 27, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 569 views
+1 vote
1 answer

Hadoop: "DEPRECATED: Use of this script to execute hdfs command is deprecated." error

The use of dfs in your command is "Deprecated". ...READ MORE

Nov 26, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 8,262 views
0 votes
1 answer

How to find the running namenodes and secondary name nodes in hadoop?

Name nodes: hdfs getconf -namenodes Secondary name nodes: hdfs getconf ...READ MORE

Nov 26, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,440 views
0 votes
1 answer

Hadoop: namenode -format error "java.io.IOException"

Run the command as sudo or add the ...READ MORE

Nov 26, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,185 views
0 votes
1 answer

How to use mountable HDFS on Centos?

Hey. You can use the following commands ...READ MORE

Nov 26, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,888 views
0 votes
1 answer

Not able to run mapreduce job. Attempt failes error

The most common reason for this is ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,014 views
0 votes
1 answer

What is the command to count number of lines in a file in hdfs?

hadoop fs -cat /example2/doc1 | wc -l READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,385 views
0 votes
1 answer

Hadoop hdfs "permission denied" error when I try to make new directory

This is because you dont have enough ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 15,011 views
0 votes
1 answer

Apache Spark gives "Failed to load native-hadoop with error"

Seems like hadoop path is missing in java.library.path. ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,476 views
+1 vote
1 answer

How to read HDFS and local files with the same code in Java?

You can try something like this: ​ ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,515 views
0 votes
1 answer

What is the usage of Configured class in Hadoop programs?

Configured is a default implementation of the Configurable interface - ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,094 views
0 votes
1 answer

Which is the easiest way for text analytics with hadoop?

Apache pig provides CSVExcelStorage class for loading ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 683 views
0 votes
1 answer

Error while connecting to Hive using Java JDBC

Use ​org.apache.hive.jdbc.HiveDriver as your driver ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,594 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 12,868 views
0 votes
1 answer

How to upgrade from Hadoop 1.2 to Hadoop 2.2?

Here's the solution that worked for me: <dependency> ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 436 views
0 votes
1 answer

How to convert Spark data into CSV?

You can use this: df.write .option("header", "true") ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,055 views
0 votes
1 answer

What is the hdfs command to check free space available?

You can see the free available space ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,313 views
0 votes
1 answer

What is SerDe in Hadoop Hive?

The SerDe interface allows you to instruct ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Akshay
 • 5,471 views
0 votes
1 answer

What is the meaning of Write Ahead Log in Hadoop?

Write Ahead Log (WAL) is a file ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Sunil
 • 1,778 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.