Hadoop Hive: recursively descending into subdirectories without partitions or editing hive-site.xml

0 votes

I run the following commands before running the Hive query to descend into the subfolders:

set hive.input.dir.recursive=true;
set hive.mapred.supports.subdirectories=true;
set hive.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

I've seen posts where people set these properties at the table-level.

TBLPROPERTIES ("hive.input.dir.recursive" = "TRUE", 
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE", 
    "mapred.input.dir.recursive" = "TRUE");

Is there any other way to do this?

Nov 15, 2018 in Big Data Hadoop by digger
• 26,600 points
1,173 views

1 answer to this question.

0 votes

 I set the following properties before I create my external table in the Hive query and it works for me.

SET hive.mapred.supports.subdirectories=TRUE;
SET mapred.input.dir.recursive=TRUE;
answered Nov 15, 2018 by Omkar
• 68,480 points

Related Questions In Big Data Hadoop

+1 vote
1 answer

Where to set hadoop.tmp.dir? core-site.xml or hdfs-site.xml?

hadoop.tmp.dir (A base for other temporary directories) is ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,800 points
2,188 views
0 votes
1 answer

Meaning of fs.defaultFS property in core-site.xml in hadoop

fs.default.name property in core-site.xml tells the address ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,350 points
1,576 views
0 votes
1 answer

Hadoop Hive Hbase: How to insert data into Hbase using Hive (JSON file)?

You can use the get_json_object function to parse the ...READ MORE

answered Nov 15, 2018 in Big Data Hadoop by Omkar
• 68,480 points
716 views
0 votes
1 answer

How to load data from flat files into different partitions of Hive table?

Hi, You can load data from flat files ...READ MORE

answered Jun 26 in Big Data Hadoop by Gitika
• 25,420 points
66 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
3,573 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,800 points
453 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,415 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,332 views
0 votes
1 answer

Hadoop Hive: How to convert multiple rows into comma separated values?

You can use the aggregator function collect_set to do ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 68,480 points
820 views
0 votes
1 answer

Hadoop Hive: How to split a single row into multiple rows?

Try this SELECT ID1, Sub FROM tableName lateral view ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 68,480 points
1,450 views