Hadoop Hive recursively descending into subdirectories without partitions or editing hive-site xml

0 votes

I run the following commands before running the Hive query to descend into the subfolders:

set hive.input.dir.recursive=true;
set hive.mapred.supports.subdirectories=true;
set hive.supports.subdirectories=true;
set mapred.input.dir.recursive=true;

I've seen posts where people set these properties at the table-level.

TBLPROPERTIES ("hive.input.dir.recursive" = "TRUE", 
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE", 
    "mapred.input.dir.recursive" = "TRUE");

Is there any other way to do this?

Nov 15, 2018 in Big Data Hadoop by digger
• 26,740 points
5,175 views

1 answer to this question.

0 votes

 I set the following properties before I create my external table in the Hive query and it works for me.

SET hive.mapred.supports.subdirectories=TRUE;
SET mapred.input.dir.recursive=TRUE;
answered Nov 15, 2018 by Omkar
• 69,210 points

Related Questions In Big Data Hadoop

+1 vote
1 answer

Where to set hadoop.tmp.dir? core-site.xml or hdfs-site.xml?

hadoop.tmp.dir (A base for other temporary directories) is ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
8,517 views
0 votes
2 answers

Meaning of fs.defaultFS property in core-site.xml in hadoop

It allows to run dfs commands more ...READ MORE

answered Jan 22, 2020 in Big Data Hadoop by MD
• 95,440 points
7,070 views
0 votes
1 answer

Hadoop Hive Hbase: How to insert data into Hbase using Hive (JSON file)?

You can use the get_json_object function to parse the ...READ MORE

answered Nov 15, 2018 in Big Data Hadoop by Omkar
• 69,210 points
2,648 views
0 votes
1 answer

How to load data from flat files into different partitions of Hive table?

Hi, You can load data from flat files ...READ MORE

answered Jun 26, 2019 in Big Data Hadoop by Gitika
• 65,910 points
1,373 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,618 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,215 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,945 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,296 views
0 votes
1 answer

Hadoop Hive: How to convert multiple rows into comma separated values?

You can use the aggregator function collect_set to do ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 69,210 points
4,955 views
0 votes
1 answer

Hadoop Hive: How to split a single row into multiple rows?

Try this SELECT ID1, Sub FROM tableName lateral view ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 69,210 points
8,617 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP