Trending questions in Big Data Hadoop

0 votes
1 answer

Extracting pig log messages

You can use the following commands in ...READ MORE

Jan 7, 2019 in Big Data Hadoop by Omkar
• 69,210 points
531 views
0 votes
1 answer

How to find the running namenodes and secondary name nodes in hadoop?

Name nodes: hdfs getconf -namenodes Secondary name nodes: hdfs getconf ...READ MORE

Nov 26, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,327 views
0 votes
1 answer

What is the difference between Hive internal tables and external tables?

Hive has a relational database on the ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Neha
• 6,300 points
2,626 views
0 votes
1 answer

Not able to start MySql service

Hello. "The system never lies :-P".  The service ...READ MORE

Dec 26, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,022 views
0 votes
1 answer

Sqoop data transfer with SSL or SFtp

Sqoop supports SSL/TLS data transfer with the ...READ MORE

Dec 18, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,366 views
0 votes
1 answer

Hadoop: java.io.IOException: File could only be replicated to 0 nodes instead of minReplication (=1)

Try this, first stop all the daemons, ...READ MORE

Nov 6, 2018 in Big Data Hadoop by Omkar
• 69,210 points
3,185 views
0 votes
1 answer

Hadoop backup and recovery tool and guidance

There are a few options for backup ...READ MORE

Jan 4, 2019 in Big Data Hadoop by Frankie
• 9,830 points
649 views
0 votes
1 answer

Issue While Running Oozie Sample Project

Hey @supriya.  Seems like you have not set ...READ MORE

Jan 4, 2019 in Big Data Hadoop by Omkar
• 69,210 points
643 views
0 votes
1 answer

Hadoop Mapreduce: Class Not Found Exception

Try adding this Job job = new Job(conf, ...READ MORE

Oct 29, 2018 in Big Data Hadoop by Omkar
• 69,210 points
3,495 views
0 votes
1 answer

Hadoop plugin (1.0.3) for eclipse

When you have Hadoop Eclipse plugin installed ...READ MORE

Dec 28, 2018 in Big Data Hadoop by Frankie
• 9,830 points
924 views
0 votes
1 answer

Apache Spark gives "Failed to load native-hadoop with error"

Seems like hadoop path is missing in java.library.path. ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,434 views
0 votes
1 answer

Hadoop sqoop import not working. Error: ERROR manager.SqlManager: Error reading from database

In the command, try mentioning the driver ...READ MORE

Dec 5, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,853 views
–1 vote
1 answer

Not able to execute mapreduce code

Follow the below steps to execute the ...READ MORE

Jan 7, 2019 in Big Data Hadoop by Omkar
• 69,210 points
457 views
0 votes
1 answer

Processing image in hdfs

If your block size is 64 MB, ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
886 views
+1 vote
1 answer

Not able to start datanode in Hadoop

Follow these steps: Stop namenode Delete the datanode directory ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,475 views
–1 vote
1 answer

Getting error when writing to HDFS. Unable to load native-hadoop library for your platform

Try this: sudo service hadoop-master restart After that try ...READ MORE

Dec 19, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,206 views
0 votes
1 answer

MySql connection problem "Cant connect to local mysql server through socket "var/lib/mysql/mysql.sock" .

First start the mysql server: service mysqld start To ...READ MORE

Dec 26, 2018 in Big Data Hadoop by Omkar
• 69,210 points
864 views
0 votes
1 answer

How are blocks created while written file in hdfs?

Suppose we want to write a 1 ...READ MORE

Dec 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,079 views
–1 vote
1 answer

Copying jar file into hadoop

hdfs dfs -put input_file_name output_location READ MORE

Dec 19, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,183 views
0 votes
1 answer

Hadoop Hive Hbase: How to insert data into Hbase using Hive (JSON file)?

You can use the get_json_object function to parse the ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,619 views
+4 votes
3 answers

Unable to find file error- while Streaming Hadoop

Is python installed running on the slaves that ...READ MORE

Oct 16, 2018 in Big Data Hadoop by Vinith
3,684 views
–1 vote
1 answer

Job History Server edit logs

When the application master fails, each file ...READ MORE

Dec 26, 2018 in Big Data Hadoop by Omkar
• 69,210 points
853 views
0 votes
1 answer

How to securely transfer data from rdms to hdfs using sqoop?

Sqoop stores metadata in a repository and ...READ MORE

Dec 18, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,152 views
0 votes
1 answer

Hive ACID semantics

ACID stands for Atomicity, Consistency, Isolation, and Durability.  Until ...READ MORE

Jan 4, 2019 in Big Data Hadoop by Omkar
• 69,210 points
411 views
0 votes
1 answer

Small files and HDFS blocks

Multiple files are not stored in a ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Frankie
• 9,830 points
2,518 views
0 votes
1 answer

What are SUCCESS and part-r-00000 files in Hadoop?

Yes, both the files i.e. SUCCESS and ...READ MORE

Apr 12, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
11,935 views
–1 vote
1 answer

Where does reduce take place in Hadoop MapReduce?

A MapReduce job usually splits the input data-set into ...READ MORE

Dec 18, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,161 views
0 votes
1 answer

What is -cp command in hadoop? How it works?

/user/cloudera/data1 is not a directory, it is ...READ MORE

Oct 17, 2018 in Big Data Hadoop by Frankie
• 9,830 points
3,814 views
–1 vote
1 answer

How to restrict nodes that talk to namenode?

Step 1: Create includes file in /home/hadoop ...READ MORE

Jan 5, 2019 in Big Data Hadoop by Omkar
• 69,210 points
419 views
0 votes
1 answer

How do I install cloudera Hadoop without internet connection?

You can do that by selecting the ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Neha
• 6,300 points
1,372 views
0 votes
1 answer

Pig CLI mode

Pig can be used in two modes: 1) ...READ MORE

Jan 4, 2019 in Big Data Hadoop by Omkar
• 69,210 points
361 views
–1 vote
1 answer

When to use Sqoop?

Sqoop is used to transfer any data ...READ MORE

Dec 31, 2018 in Big Data Hadoop by Omkar
• 69,210 points
574 views
0 votes
1 answer

Hadoop Mapreduce: Error: Could not find or load main class com.sun.tools.javac.Main

You have to add HADOOP_CLASSPATH environment parameter: expor ...READ MORE

Oct 30, 2018 in Big Data Hadoop by Omkar
• 69,210 points
3,191 views
–1 vote
1 answer

Issue with MySql Server connection

Try to restart the mysqld server and then login: sudo ...READ MORE

Jan 2, 2019 in Big Data Hadoop by Omkar
• 69,210 points
449 views
–1 vote
1 answer

How to Set cluster Rebalancer threshold

The command you are using is wrong. ...READ MORE

Jan 5, 2019 in Big Data Hadoop by Omkar
• 69,210 points
318 views
–1 vote
1 answer

How to start pyspark?

Follow these steps: First start hadoop daemons: cd $HADOOP_HOME/sbin ./start-all.sh Now ...READ MORE

Jan 3, 2019 in Big Data Hadoop by Omkar
• 69,210 points
361 views
0 votes
1 answer

How can I calculate exact median with Apache Spark?

You need to sort RDD and take ...READ MORE

Oct 8, 2018 in Big Data Hadoop by Omkar
• 69,210 points
4,047 views
0 votes
1 answer

How to execute wordcount in Hadoop?

Follow these steps: Step 1:  Import all these hadoop ...READ MORE

Dec 19, 2018 in Big Data Hadoop by Omkar
• 69,210 points
924 views
–1 vote
1 answer

Serde input and output

Input Processing Hive's execution engine (referred to as ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
615 views
+1 vote
1 answer

What is difference between Hadoop Namenode HA and HDFS federation?

The main difference between HDFS High Availability ...READ MORE

Oct 31, 2018 in Big Data Hadoop by Frankie
• 9,830 points
3,027 views
–1 vote
1 answer

File is available at HDFS location. Still system throws en error.

This is happening because the file name ...READ MORE

Jan 2, 2019 in Big Data Hadoop by Omkar
• 69,210 points
329 views
0 votes
1 answer

Hadoop Spark: __spark_lib.zip does not exist

To find this file, your HADOOP_CONF_DIR env ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,359 views
–1 vote
1 answer

Mapreduce in Python

mapper.py #!/usr/bin/python import sys #Word Count Example # input comes from ...READ MORE

Dec 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
848 views
–1 vote
1 answer

Spark multiple version without major version

Yes. It is not necessary to set ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
535 views
0 votes
1 answer

How is a file written in hdfs?

When you copy a file from the ...READ MORE

Dec 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
752 views
0 votes
1 answer

How to read Spark elements having multiple lines each?

Try this: val new_records = sc.newAPIHadoopRDD(hadoopConf,classOf[ ...READ MORE

Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,125 views
0 votes
1 answer

Can you build “Spark” with any particular Hadoop version?

Yes, one can build “Spark” for a specific ...READ MORE

Dec 14, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,049 views
0 votes
1 answer

Hadoop: Adding multiple inputs with a different mapper using python

hadoop jar hadoop-multiple-streaming.jar \ ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,116 views
–1 vote
1 answer

Hadoop cluster is not running in vm

First check if all daemons are running: sudo ...READ MORE

Dec 26, 2018 in Big Data Hadoop by Omkar
• 69,210 points
557 views
0 votes
1 answer

Mapreduce shuffle and sort

The mapreduce task happens in the following ...READ MORE

Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
471 views