Joining Multiple Spark Dataframes

0 votes
I have 'n' Spark Data-frames like this:

list(d1,d2,d3,d4,.....dn)

How can i join these 'n' Data-frames together?
Mar 26, 2018 in Big Data Hadoop by Ashish
• 2,650 points
2,618 views

1 answer to this question.

0 votes

You can run the below code to join those 'n' spark data-frames:

List(df1,df2,df3,dfN).reduce((a, b) => a.join(b, joinCondition))
answered Mar 26, 2018 by Bharani
• 4,660 points
Hi Bharani, Can you please explain a little bit. How are trying to achieve this.

Related Questions In Big Data Hadoop

0 votes
1 answer

How to read Spark elements having multiple lines each?

Try this: val new_records = sc.newAPIHadoopRDD(hadoopConf,classOf[ ...READ MORE

answered Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,125 views
–1 vote
1 answer

How to use multiple spark version?

You can use the  SPARK_MAJOR_VERSION for this. Suppose ...READ MORE

answered Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
1,908 views
–1 vote
1 answer

Spark multiple version without major version

Yes. It is not necessary to set ...READ MORE

answered Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,210 points
535 views
0 votes
1 answer

How to merge two Spark DataFrames?

Hi@akhtar, You can perform join operation in spark. ...READ MORE

answered Sep 17, 2020 in Big Data Hadoop by MD
• 95,440 points
1,169 views
0 votes
1 answer

How to sync Hadoop configuration files to multiple nodes?

For syncing Hadoop configuration files, you have ...READ MORE

answered Jun 21, 2018 in Big Data Hadoop by HackTheCode
1,166 views
+1 vote
1 answer
+1 vote
2 answers
0 votes
1 answer

What do we exactly mean by “Hadoop” – the definition of Hadoop?

The official definition of Apache Hadoop given ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by Shubham
1,602 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,523 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,166 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP