Hadoop Hive: recursively descending into subdirectories without partitions or editing hive-site.xml

0 votes

I run the following commands before running the Hive query to descend into the subfolders:

set hive.input.dir.recursive=true;
set hive.mapred.supports.subdirectories=true;
set hive.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

I've seen posts where people set these properties at the table-level.

TBLPROPERTIES ("hive.input.dir.recursive" = "TRUE", 
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE", 
    "mapred.input.dir.recursive" = "TRUE");

Is there any other way to do this?

Nov 15, 2018 in Big Data Hadoop by digger
• 27,620 points
577 views

1 answer to this question.

0 votes

 I set the following properties before I create my external table in the Hive query and it works for me.

SET hive.mapred.supports.subdirectories=TRUE;
SET mapred.input.dir.recursive=TRUE;
answered Nov 15, 2018 by Omkar
• 67,120 points

Related Questions In Big Data Hadoop

+1 vote
1 answer

Where to set hadoop.tmp.dir? core-site.xml or hdfs-site.xml?

hadoop.tmp.dir (A base for other temporary directories) is ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,070 points
1,519 views
0 votes
1 answer

Meaning of fs.defaultFS property in core-site.xml in hadoop

fs.default.name property in core-site.xml tells the address ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,110 points
990 views
0 votes
1 answer

Hadoop Hive Hbase: How to insert data into Hbase using Hive (JSON file)?

You can use the get_json_object function to parse the ...READ MORE

answered Nov 15, 2018 in Big Data Hadoop by Omkar
• 67,120 points
407 views
0 votes
1 answer

How to load data from flat files into different partitions of Hive table?

Hi, You can load data from flat files ...READ MORE

answered Jun 26 in Big Data Hadoop by Gitika
• 19,720 points
26 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,070 points
2,040 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,070 points
193 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,410 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
762 views
0 votes
1 answer

Hadoop Hive: How to convert multiple rows into comma separated values?

You can use the aggregator function collect_set to do ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 67,120 points
411 views
0 votes
1 answer

Hadoop Hive: How to split a single row into multiple rows?

Try this SELECT ID1, Sub FROM tableName lateral view ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 67,120 points
656 views