Latest questions in Big Data Hadoop

+1 vote
1 answer

How to read HDFS and local files with the same code in Java?

You can try something like this: ​ ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,514 views
0 votes
1 answer

What is the usage of Configured class in Hadoop programs?

Configured is a default implementation of the Configurable interface - ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 1,092 views
0 votes
1 answer

Which is the easiest way for text analytics with hadoop?

Apache pig provides CSVExcelStorage class for loading ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 683 views
0 votes
1 answer

Error while connecting to Hive using Java JDBC

Use ​org.apache.hive.jdbc.HiveDriver as your driver ...READ MORE

Nov 22, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,593 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 12,868 views
0 votes
1 answer

How to upgrade from Hadoop 1.2 to Hadoop 2.2?

Here's the solution that worked for me: <dependency> ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 434 views
0 votes
1 answer

How to convert Spark data into CSV?

You can use this: df.write .option("header", "true") ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,054 views
0 votes
1 answer

What is the hdfs command to check free space available?

You can see the free available space ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,309 views
0 votes
1 answer

What is SerDe in Hadoop Hive?

The SerDe interface allows you to instruct ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Akshay
 • 5,471 views
0 votes
1 answer

What is the meaning of Write Ahead Log in Hadoop?

Write Ahead Log (WAL) is a file ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Sunil
 • 1,777 views
0 votes
1 answer

Hadoop: What does RDD mean?

RDD is the acronym for Resilient Distribution ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Deepak
 • 940 views
0 votes
1 answer

What is the difference between Apache ZooKeeper and Apache Oozie?

The main difference between these two is ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Anoop
 • 4,061 views
0 votes
1 answer

Hadoop: How to get the column name along with the output in Hive?

You can get the column names by ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,650 views
0 votes
1 answer

Hadoop: Java code to get Hive table properties.

You can try this: org.apache.hadoop.hive.metastore.api.Table table=..........; org.apache.hadoop.hive.metastore.api.StorageDescriptor sd=table.getSd(); String loc=sd.getLocation(); String ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 717 views
0 votes
1 answer

Hadoop Java: org.apache.hadoop.conf.Configuration does not exist error

To solve this error, add hadoop-common to ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 7,095 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 770 views
0 votes
1 answer

Hadoop HDFS: Exception in createBlockOutputStream

It's a connection error. Have you turned ...READ MORE

Nov 21, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,690 views
0 votes
1 answer

Hadoop: "Caused by: java.lang.ClassNotFoundException" error

Have you placed the jar files in ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 6,632 views
0 votes
1 answer

Hadoop hdfs: How to count number of lines?

You can count the number of lines ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 960 views
0 votes
1 answer

Hadoop: Adding multiple inputs with a different mapper using python

hadoop jar hadoop-multiple-streaming.jar \ ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,193 views
0 votes
1 answer

Hadoop HDFS: Java API to move files to hdfs

You can use the FileUtil api to do this. Example: Configuration ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,804 views
0 votes
1 answer

Hadoop Apache: Could not fully initialize service [org.apache.oozie.service.ShareLibService]

As mentioned in the error, the problem ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 913 views
0 votes
1 answer

What is the difference between Hive internal tables and external tables?

Hive has a relational database on the ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 2,661 views
0 votes
1 answer

How to use Sqoop in Java Program?

You can run sqoop from inside your ...READ MORE

Nov 19, 2018 in Big Data Hadoop by Neha
 • 6,300 points • 1,523 views
0 votes
1 answer

Hadoop MapReduce intermediate output

keep.task.files.pattern parameter can be used to keep the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 573 views
0 votes
1 answer

Small files and HDFS blocks

Multiple files are not stored in a ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Frankie
 • 9,830 points • 2,560 views
0 votes
1 answer

Hadoop: Error: Could not find or load main class org.apache.hadoop.util.VersionInfo

This seems like a path issue. Add the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 11,976 views
0 votes
1 answer

Hadoop with Python: How to use python to stop Hadoop tasktracker?

Here you go os.system('sudo -u hadoop bin/hadoop-daemon.sh stop ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 380 views
0 votes
1 answer

Hadoop Pig: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/filter/Filter

This seems like a problem with the ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,872 views
0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 847 views
0 votes
1 answer

Hadoop Spark Derby: Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database

As mentioned in the error, there's another ...READ MORE

Nov 16, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 4,369 views
0 votes
1 answer

Can someone tell me how apache sqoop works?

This blog would give you answers to ...READ MORE

Nov 16, 2018 in Big Data Hadoop by brat_1
 • 7,200 points • 422 views
0 votes
1 answer

how is apache sqoop different from flume? Can someone explain?

Apache Sqoop lets you import and export ...READ MORE

Nov 15, 2018 in Big Data Hadoop by brat_1
 • 7,200 points • 354 views
+2 votes
1 answer

Hadoop: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable

Add these 2 lines in your code ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 6,059 views
0 votes
1 answer

Hadoop Hive Hbase: How to insert data into Hbase using Hive (JSON file)?

You can use the get_json_object function to parse the ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,668 views
0 votes
1 answer

Hadoop HDFS: How to delete old files from HDFS?

You can use commands like this: hdfs dfs ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 13,607 views
0 votes
1 answer

Hadoop Spark: __spark_lib.zip does not exist

To find this file, your HADOOP_CONF_DIR env ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 2,451 views
0 votes
1 answer

Hadoop Hive: recursively descending into subdirectories without partitions or editing hive-site.xml

 I set the following properties before I ...READ MORE

Nov 15, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,221 views
0 votes
1 answer

Hadoop: How to copy directory from local system to hdfs using Java code?

Just use the FileSystem's copyFromLocalFile method. If the source Path ...READ MORE

Nov 14, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,343 views
0 votes
1 answer

Hadoop Hive: How to drop a table?

You cannot drop column directly from a ...READ MORE

Nov 14, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,809 views
0 votes
1 answer

Hadoop Hive: How to split a single row into multiple rows?

Try this SELECT ID1, Sub FROM tableName lateral view ...READ MORE

Nov 14, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 8,703 views
0 votes
1 answer

Hadoop Hive: How to convert multiple rows into comma separated values?

You can use the aggregator function collect_set to do ...READ MORE

Nov 14, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,005 views
0 votes
1 answer

Hadoop: RHadoop: packages ‘rmr’, ‘rJava‘, ‘RJSONIO‘, ‘rhdfs’, ‘rhbase’, ‘plyrmr’ are not available

You need to first download the packages ...READ MORE

Nov 14, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 830 views
0 votes
1 answer

Hadoop MacOS: formatting namenode: Could not find or load main class

You need to point the HADOOP_PREFIX to ...READ MORE

Nov 13, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,013 views
+1 vote
2 answers

Hadoop Hive: Not recognizing alias named in select part?

You can not use column aliases in ...READ MORE

Nov 26, 2019 in Big Data Hadoop by anonymous
 • 10,399 views
0 votes
1 answer

Hadoop Spark: What is version to find SparkSession in library Spark?

you need both core and SQL artifacts <repositories> ...READ MORE

Nov 13, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,797 views
0 votes
1 answer

Hadoop: method to send output to multiple directories

setup: MultipleOutputs.addNamedOutput(job, "Output", TextOutputFormat.class, Text.class, Text.class); setup of reducer: mout ...READ MORE

Nov 13, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 755 views
0 votes
1 answer

Hadoop Sqoop: How to change sqoop metastore?

In sqoop-site.xml I configured it with, the ...READ MORE

Nov 13, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 1,525 views
0 votes
1 answer

Hadoop: Format namenode gives error: Shutting down NameNode at java.net.UnknownHostException

UnknownHostException is thrown when hadoop tries to resolve ...READ MORE

Nov 12, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 5,364 views
0 votes
1 answer

Hadoop Hive: message:Version information not found in metastore. Unable to instantiate

These are the necessary tables required for metastore that are ...READ MORE

Nov 12, 2018 in Big Data Hadoop by Omkar
 • 69,230 points • 3,324 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.